Improving xmlrpc introspection

[Claudiu Popa]

Hello.

This idea proposes enhancing the xmlrpc library by adding a couple
of introspectable servers and proxies. For instance, here's an output of
using the current idioms.

proxy = ServerProxy('http://localhost:8000')
dir(proxy)

['_ServerProxy__allow_none', '_ServerProxy__close',
'_ServerProxy__encoding', '_ServerProxy__handler',
'_ServerProxy__host', '_ServerProxy__request',
'_ServerProxy__transport', '_ServerProxy__verbose', '__call__',
'__class__', '__delattr__', '__dict__', '__dir__', '__doc__',
'__enter__', '__eq__', '__exit__', '__format__', '__ge__',
'__getattr__'
, '__getattribute__', '__gt__', '__hash__', '__init__', '__le__',
'__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__']


Nothing useful in dir. The following works only if the server enables

proxy.system.listMethods()

['mul', 'pow', 'system.listMethods', 'system.methodHelp',
'system.methodSignature']

proxy.mul

<xmlrpc.client._Method object at 0x02AFB690>

help(proxy.mul)

Help on _Method in module xmlrpc.client object:

class _Method(builtins.object)
| Methods defined here:
|
| __call__(self, *args)
|
| __getattr__(self, name)
|
| __init__(self, send, name)
| # some magic to bind an XML-RPC method to an RPC server.
| # supports "nested" methods (e.g. examples.getStateName)
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)

proxy.system.methodHelp('mul')

'multiplication'

proxy.system.methodSignature('mul')

'signatures not supported'


We can find out something about that method by calling it.

proxy.mul(1, 2, 3)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1091, in __call__
return self.__send(self.__name, args)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1421, in __request
verbose=self.__verbose
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1133, in request
return self.single_request(host, handler, request_body, verbose)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1149, in single_request
return self.parse_response(resp)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1320, in parse_response
return u.close()
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 658, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault 1: "<class 'TypeError'>:mul() takes 3
positional arguments but 4 were given">


So, only after calling a method, one can find meaningful informations about it.

from xmlrpc.client import MagicProxy # not a very good name, but it does some magic behind
proxy = MagicProxy('http://localhost:8000')
dir(proxy)

['_ServerProxy__allow_none', '_ServerProxy__close',
'_ServerProxy__encoding', '_ServerProxy__handler',
'_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__trans
', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__',
'__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__',
'__format__', '__ge__',
'__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__',
'__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__',
'_collect_methods', '_original_mul', '_original_pow', 'mul', 'pow']

proxy.mul

<function mul at 0x035AD5D8>

proxy.pow

<function pow at 0x035AD638>

help(proxy.mul)

Help on function mul in module xmlrpc.client:

mul(x:1, y) -> 2
multiplication

help(proxy.pow)

Help on function pow in module xmlrpc.client:

pow(*args, **kwargs)
pow(x, y[, z]) -> number

With two arguments, equivalent to x**y. With three arguments,
equivalent to (x**y) % z, but may be more efficient (e.g. for ints).

proxy.mul(1)

Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: mul() missing 1 required positional argument: 'y'

proxy.mul(1, 2, 3)

Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: mul() takes 2 positional arguments but 3 were given

proxy.mul(1, 2)

2

import inspect
inspect.signature(proxy.mul)

<Signature at 0x35d4b98 "(x:1, y) -> 2">
As we can see, the registered methods can be introspected and calling
one with the wrong number of arguments will not trigger a request to
the server, but will fail right in the user's code.
As a problem, it will work only for servers written in Python. For
others will fallback to the current idiom.
Would something like this be useful as an addition to the stdlib's
xmlrpc module?
If someone wants to test it, here's a rough patch against tip:
https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff.

Thanks!

[Guido van Rossum]

Since this is an internet client+server, can you please also consider
security as part of your design? Perhaps it's not always a good idea to
have that much introspectability on a web interface.

Post by Claudiu Popa
Hello.
This idea proposes enhancing the xmlrpc library by adding a couple
of introspectable servers and proxies. For instance, here's an output of
using the current idioms.

proxy = ServerProxy('http://localhost:8000')
dir(proxy)

['_ServerProxy__allow_none', '_ServerProxy__close',
'_ServerProxy__encoding', '_ServerProxy__handler',
'_ServerProxy__host', '_ServerProxy__request',
'_ServerProxy__transport', '_ServerProxy__verbose', '__call__',
'__class__', '__delattr__', '__dict__', '__dir__', '__doc__',
'__enter__', '__eq__', '__exit__', '__format__', '__ge__',
'__getattr__'
, '__getattribute__', '__gt__', '__hash__', '__init__', '__le__',
'__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__']
Nothing useful in dir. The following works only if the server enables

proxy.system.listMethods()

['mul', 'pow', 'system.listMethods', 'system.methodHelp',
'system.methodSignature']

proxy.mul

<xmlrpc.client._Method object at 0x02AFB690>

help(proxy.mul)

class _Method(builtins.object)
|
| __call__(self, *args)
|
| __getattr__(self, name)
|
| __init__(self, send, name)
| # some magic to bind an XML-RPC method to an RPC server.
| # supports "nested" methods (e.g. examples.getStateName)
|
| ----------------------------------------------------------------------
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)

proxy.system.methodHelp('mul')

'multiplication'

proxy.system.methodSignature('mul')

'signatures not supported'
We can find out something about that method by calling it.

proxy.mul(1, 2, 3)

File "<stdin>", line 1, in <module>
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1091, in __call__
return self.__send(self.__name, args)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1421, in __request
verbose=self.__verbose
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1133, in request
return self.single_request(host, handler, request_body, verbose)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1149, in single_request
return self.parse_response(resp)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1320, in parse_response
return u.close()
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 658, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault 1: "<class 'TypeError'>:mul() takes 3
positional arguments but 4 were given">
So, only after calling a method, one can find meaningful informations about it.

from xmlrpc.client import MagicProxy # not a very good name, but it

does some magic behind

proxy = MagicProxy('http://localhost:8000')
dir(proxy)

['_ServerProxy__allow_none', '_ServerProxy__close',
'_ServerProxy__encoding', '_ServerProxy__handler',
'_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__trans
', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__',
'__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__',
'__format__', '__ge__',
'__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__',
'__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__',
'_collect_methods', '_original_mul', '_original_pow', 'mul', 'pow']

proxy.mul

<function mul at 0x035AD5D8>

proxy.pow

<function pow at 0x035AD638>

help(proxy.mul)

mul(x:1, y) -> 2
multiplication

help(proxy.pow)

pow(*args, **kwargs)
pow(x, y[, z]) -> number
With two arguments, equivalent to x**y. With three arguments,
equivalent to (x**y) % z, but may be more efficient (e.g. for ints).

proxy.mul(1)

File "<console>", line 1, in <module>
TypeError: mul() missing 1 required positional argument: 'y'

proxy.mul(1, 2, 3)

File "<console>", line 1, in <module>
TypeError: mul() takes 2 positional arguments but 3 were given

proxy.mul(1, 2)

2

import inspect
inspect.signature(proxy.mul)

<Signature at 0x35d4b98 "(x:1, y) -> 2">
As we can see, the registered methods can be introspected and calling
one with the wrong number of arguments will not trigger a request to
the server, but will fail right in the user's code.
As a problem, it will work only for servers written in Python. For
others will fallback to the current idiom.
Would something like this be useful as an addition to the stdlib's
xmlrpc module?
https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff.
Thanks!
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Nick Coghlan]

Post by Guido van Rossum
Since this is an internet client+server, can you please also consider

security as part of your design? Perhaps it's not always a good idea to
have that much introspectability on a web interface.

I don't recall the details, but there was a CVE quite some time ago for an
information leak in SimpleXMLRPCServer. That's not necessarily a "this is a
bad idea" response, just "the security implications of such a feature would
need to be managed very carefully (and if that's too hard to do, it might
be a bad idea)".

Cheers,
Nick.

Post by Guido van Rossum

Post by Claudiu Popa
Hello.
This idea proposes enhancing the xmlrpc library by adding a couple
of introspectable servers and proxies. For instance, here's an output of
using the current idioms.

proxy = ServerProxy('http://localhost:8000')
dir(proxy)

['_ServerProxy__allow_none', '_ServerProxy__close',
'_ServerProxy__encoding', '_ServerProxy__handler',
'_ServerProxy__host', '_ServerProxy__request',
'_ServerProxy__transport', '_ServerProxy__verbose', '__call__',
'__class__', '__delattr__', '__dict__', '__dir__', '__doc__',
'__enter__', '__eq__', '__exit__', '__format__', '__ge__',
'__getattr__'
, '__getattribute__', '__gt__', '__hash__', '__init__', '__le__',
'__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__']
Nothing useful in dir. The following works only if the server enables

proxy.system.listMethods()

['mul', 'pow', 'system.listMethods', 'system.methodHelp',
'system.methodSignature']

proxy.mul

<xmlrpc.client._Method object at 0x02AFB690>

help(proxy.mul)

class _Method(builtins.object)
|
| __call__(self, *args)
|
| __getattr__(self, name)
|
| __init__(self, send, name)
| # some magic to bind an XML-RPC method to an RPC server.
| # supports "nested" methods (e.g. examples.getStateName)
|
|

----------------------------------------------------------------------

Post by Guido van Rossum

Post by Claudiu Popa
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)

proxy.system.methodHelp('mul')

'multiplication'

proxy.system.methodSignature('mul')

'signatures not supported'
We can find out something about that method by calling it.

proxy.mul(1, 2, 3)

File "<stdin>", line 1, in <module>
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1091, in __call__
return self.__send(self.__name, args)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1421, in __request
verbose=self.__verbose
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1133, in request
return self.single_request(host, handler, request_body, verbose)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1149, in single_request
return self.parse_response(resp)
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1320, in parse_response
return u.close()
File "D:\Projects\cpython\lib\xmlrpc\client.py", line 658, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault 1: "<class 'TypeError'>:mul() takes 3
positional arguments but 4 were given">
So, only after calling a method, one can find meaningful informations about it.

from xmlrpc.client import MagicProxy # not a very good name, but it

does some magic behind

Post by Guido van Rossum

Post by Claudiu Popa

proxy = MagicProxy('http://localhost:8000')
dir(proxy)

['_ServerProxy__allow_none', '_ServerProxy__close',
'_ServerProxy__encoding', '_ServerProxy__handler',
'_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__trans
', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__',
'__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__',
'__format__', '__ge__',
'__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__',
'__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__',
'_collect_methods', '_original_mul', '_original_pow', 'mul', 'pow']

proxy.mul

<function mul at 0x035AD5D8>

proxy.pow

<function pow at 0x035AD638>

help(proxy.mul)

mul(x:1, y) -> 2
multiplication

help(proxy.pow)

pow(*args, **kwargs)
pow(x, y[, z]) -> number
With two arguments, equivalent to x**y. With three arguments,
equivalent to (x**y) % z, but may be more efficient (e.g. for ints).

proxy.mul(1)

File "<console>", line 1, in <module>
TypeError: mul() missing 1 required positional argument: 'y'

proxy.mul(1, 2, 3)

File "<console>", line 1, in <module>
TypeError: mul() takes 2 positional arguments but 3 were given

proxy.mul(1, 2)

2

import inspect
inspect.signature(proxy.mul)

<Signature at 0x35d4b98 "(x:1, y) -> 2">
As we can see, the registered methods can be introspected and calling
one with the wrong number of arguments will not trigger a request to
the server, but will fail right in the user's code.
As a problem, it will work only for servers written in Python. For
others will fallback to the current idiom.
Would something like this be useful as an addition to the stdlib's
xmlrpc module?
https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff.
Thanks!
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Skip Montanaro]

I might be a bit confused (nothing new there), but it seemed to me

Post by Claudiu Popa
As we can see, the registered methods can be introspected and calling
one with the wrong number of arguments will not trigger a request to
the server, but will fail right in the user's code.

I think we will have to see the code to decide if it's a security
risk. Claudiu, I suggest you open an issue in the tracker so others
can see how the magic works.

Skip

[Claudiu Popa]

Received: from localhost (HELO mail.python.org) (127.0.0.1)
by albatross.python.org with SMTP; 19 Jun 2014 08:35:34 +0200
Received: from mail-qc0-x233.google.com (unknown
[IPv6:2607:f8b0:400d:c01::233])
(using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits))
(No client certificate requested)
by mail.python.org (Postfix) with ESMTPS
for <python-ideas-+ZN9ApsXKcEdnm+***@public.gmane.org>; Thu, 19 Jun 2014 08:35:33 +0200 (CEST)
Received: by mail-qc0-f179.google.com with SMTP id x3so1771324qcv.24
for <python-ideas-+ZN9ApsXKcEdnm+***@public.gmane.org>; Wed, 18 Jun 2014 23:35:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc:content-type;
bh=SLEANKUND69r4HH0I2/6gb+TVtEeqALfHo6ElRcENHg=;
b=YU22mNyWrAPP2oHJuB0U44kUU1RbhR4jd57jOs5q71hF1If8ysx6bQ3zbzvD0DQuq6
1gBTmQ5cyp4IRAvH+L1QnNcit+GneTMpYerjL0vyclxrsJKqdaLv/OBGThJ2G5MDJsuy
Z7E1cUhNYzu5qxrqatJj37tjc2kGtDshg1zuq9RpJKJe21ZjD7xekziCVUG50VbNzuPT
Zib37grjGEtw2QRtHuz0XN/L8pNsxOZ2KlrgUbOWBI5HSUB0+nfSMDrAxI/eNWD2g7Nj
asvbUXREMJ7/xiZrf4wK+CAmUF6RbCqYCmBe/qibGfYWsbLpdwt1Kj2TudF18cY0gvHN
gTcA==
X-Received: by 10.224.47.77 with SMTP id m13mr3899081qaf.69.1403159726813;
Wed, 18 Jun 2014 23:35:26 -0700 (PDT)
Received: by 10.224.175.68 with HTTP; Wed, 18 Jun 2014 23:35:26 -0700 (PDT)
In-Reply-To: <CANc-5UxoaVGCLtKTFV8+3YVt=_chxni+d2dw2sSHt19QcBaTAg-JsoAwUIsXosN+***@public.gmane.org>
X-BeenThere: python-ideas-+ZN9ApsXKcEdnm+***@public.gmane.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Discussions of speculative Python language ideas
<python-ideas.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-ideas>,
<mailto:python-ideas-request-+ZN9ApsXKcEdnm+***@public.gmane.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-ideas/>
List-Post: <mailto:python-ideas-+ZN9ApsXKcEdnm+***@public.gmane.org>
List-Help: <mailto:python-ideas-request-+ZN9ApsXKcEdnm+***@public.gmane.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-ideas>,
<mailto:python-ideas-request-+ZN9ApsXKcEdnm+***@public.gmane.org?subject=subscribe>
Errors-To: python-ideas-bounces+gcpi-python-ideas=m.gmane.org-+ZN9ApsXKcEdnm+***@public.gmane.org
Sender: "Python-ideas"
<python-ideas-bounces+gcpi-python-ideas=m.gmane.org-+ZN9ApsXKcEdnm+***@public.gmane.org>
Archived-At: <http://permalink.gmane.org/gmane.comp.python.ideas/28147>

Post by Skip Montanaro
I might be a bit confused (nothing new there), but it seemed to me

Post by Claudiu Popa
As we can see, the registered methods can be introspected and calling
one with the wrong number of arguments will not trigger a request to
the server, but will fail right in the user's code.

I think we will have to see the code to decide if it's a security
risk. Claudiu, I suggest you open an issue in the tracker so others
can see how the magic works.
Skip
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

That's right, the behaviour occurs in the client, the only catch is
that it needs
a new method in xmlrpc.server and the server must support introspection already,
by providing the `system` proxy methods.
I already posted a sample patch in the first message
(https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff).

Now, something is wrong in the client, because it exec's the
information received in
order to create the local functions, but probably there are other methods
for achieving the same behaviour.

Anyway, thank you all for your responses. I admit that I didn't think at
the security implications of this proposal very much and it was
enlightening as is.