Thread: BUG: Maintenance feature broken ?

Hi all.


When trying to use any of the maintenance features (right click on table 
-> maintenance) i get a "Maintenance job creation failed." popup.

~/.pgadmin/pgadmin4.log says:

|2018-11-05 14:35:43,015: ERROR    werkzeug:    Error on request:||
||Traceback (most recent call last):||
||  File "/usr/lib/python3.7/site-packages/werkzeug/serving.py", line 
270, in run_wsgi||
||    execute(self.server.app)||
||  File "/usr/lib/python3.7/site-packages/werkzeug/serving.py", line 
258, in execute||
||    application_iter = app(environ, start_response)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1997, in __call__||
||    return self.wsgi_app(environ, start_response)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1985, in wsgi_app||
||    response = self.handle_exception(e)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1540, in handle_exception||
||    reraise(exc_type, exc_value, tb)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/_compat.py", 
line 33, in reraise||
||    raise value||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1982, in wsgi_app||
||    response = self.full_dispatch_request()||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1614, in full_dispatch_request||
||    rv = self.handle_user_exception(e)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1517, in handle_user_exception||
||    reraise(exc_type, exc_value, tb)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/_compat.py", 
line 33, in reraise||
||    raise value||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1612, in full_dispatch_request||
||    rv = self.dispatch_request()||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py", line 
1598, in dispatch_request||
||    return self.view_functions[rule.endpoint](**req.view_args)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask_login.py", 
line 792, in decorated_view||
||    return func(*args, **kwargs)||
||  File "/usr/lib/pgadmin4/web/pgadmin/tools/maintenance/__init__.py", 
line 181, in create_maintenance_job||
||    data = json.loads(data['data'][0], encoding='utf-8')||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/simplejson/__init__.py", 
line 535, in loads||
||    return cls(encoding=encoding, **kw).decode(s)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/simplejson/decoder.py", 
line 370, in decode||
||    obj, end = self.raw_decode(s)||
||  File 
"/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/simplejson/decoder.py", 
line 400, in raw_decode||
||    return self.scan_once(s, idx=_w(s, idx).end())||
||simplejson.errors.JSONDecodeError: Expecting property name enclosed in 
double quotes or '}': line 1 column 2 (char 1)|

I tracked this bug down and it looks like the reason is the 
'data['data'][0]' in 
https://github.com/postgres/pgadmin4/blob/master/web/pgadmin/tools/maintenance/__init__.py#L181 
.

The [0] doesn't really make any sense there, and after removing it the 
problem was fixed.


PGadmin4 version: 3.4

Browser: 63.0.1 (64-bit)

(Altho i don't think it matters in this case)

This happens for me regardless of settings/tables used, so its a bit 
hard to create sample data for this.

Thus a bit more detailed:

- Right click on a table

- Select maintenance

- Click "Ok"


On my system that opens a popup with the text: "Maintenance job creation 
failed."

And IMHO that all makes sense as the request that is being posted from 
pgAdmin frontend is:


data={"op":"VACUUM","vacuum_full":false,"vacuum_freeze":false,"vacuum_analyze":false,"verbose":true,"database":"somedb","schema":"public","table":"images"}

and the relevant lines in the pgAdmin backend are 
(https://github.com/postgres/pgadmin4/blob/master/web/pgadmin/tools/maintenance/__init__.py#L181)

     data = dict(request.form)

     data = json.loads(data['data'][0], encoding='utf-8')


And thus json.loads only gets the first character of the data field ('{').

Please let me know which other infos i could provide to hunt this bug down.

Thanks,

SleepProgger


On 06.11.18 06:36, Khushboo Vashi wrote:
> Hi,
>
> I am not able to reproduce this issue at my end.
> Can you please provide some sample data through which I can reproduce 
> this issue?
>
> Thanks,
> Khushboo
>
> On Mon, Nov 5, 2018 at 11:14 PM SleepProgger <security@gnutp.com 
> <mailto:security@gnutp.com>> wrote:
>
>     Hi all.
>
>
>     When trying to use any of the maintenance features (right click on
>     table
>     -> maintenance) i get a "Maintenance job creation failed." popup.
>
>     ~/.pgadmin/pgadmin4.log says:
>
>     |2018-11-05 14:35:43,015: ERROR    werkzeug:    Error on request:||
>     ||Traceback (most recent call last):||
>     ||  File "/usr/lib/python3.7/site-packages/werkzeug/serving.py", line
>     270, in run_wsgi||
>     ||    execute(self.server.app)||
>     ||  File "/usr/lib/python3.7/site-packages/werkzeug/serving.py", line
>     258, in execute||
>     ||    application_iter = app(environ, start_response)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1997, in __call__||
>     ||    return self.wsgi_app(environ, start_response)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1985, in wsgi_app||
>     ||    response = self.handle_exception(e)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1540, in handle_exception||
>     ||    reraise(exc_type, exc_value, tb)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/_compat.py",
>
>     line 33, in reraise||
>     ||    raise value||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1982, in wsgi_app||
>     ||    response = self.full_dispatch_request()||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1614, in full_dispatch_request||
>     ||    rv = self.handle_user_exception(e)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1517, in handle_user_exception||
>     ||    reraise(exc_type, exc_value, tb)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/_compat.py",
>
>     line 33, in reraise||
>     ||    raise value||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1612, in full_dispatch_request||
>     ||    rv = self.dispatch_request()||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask/app.py",
>     line
>     1598, in dispatch_request||
>     ||    return self.view_functions[rule.endpoint](**req.view_args)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/flask_login.py",
>     line 792, in decorated_view||
>     ||    return func(*args, **kwargs)||
>     ||  File
>     "/usr/lib/pgadmin4/web/pgadmin/tools/maintenance/__init__.py",
>     line 181, in create_maintenance_job||
>     ||    data = json.loads(data['data'][0], encoding='utf-8')||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/simplejson/__init__.py",
>
>     line 535, in loads||
>     ||    return cls(encoding=encoding, **kw).decode(s)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/simplejson/decoder.py",
>
>     line 370, in decode||
>     ||    obj, end = self.raw_decode(s)||
>     ||  File
>     "/usr/lib/pgadmin4/venv/lib/python3.7/site-packages/simplejson/decoder.py",
>
>     line 400, in raw_decode||
>     ||    return self.scan_once(s, idx=_w(s, idx).end())||
>     ||simplejson.errors.JSONDecodeError: Expecting property name
>     enclosed in
>     double quotes or '}': line 1 column 2 (char 1)|
>
>     I tracked this bug down and it looks like the reason is the
>     'data['data'][0]' in
>     https://github.com/postgres/pgadmin4/blob/master/web/pgadmin/tools/maintenance/__init__.py#L181
>
>     .
>
>     The [0] doesn't really make any sense there, and after removing it
>     the
>     problem was fixed.
>
>
>     PGadmin4 version: 3.4
>
>     Browser: 63.0.1 (64-bit)
>
>     (Altho i don't think it matters in this case)
>
>

Since updating to pgAdmin4 3.6 i can't use the SSH tunnel feature anymore.

As soon as i provide the password the following exception occurs:


2019-01-03 15:36:23,007: ERROR    flask.app:    Object type <class 
'str'> cannot be passed to C code
Traceback (most recent call last):
   File 
"/usr/lib/pgadmin4/web/pgadmin/utils/driver/psycopg2/server_manager.py", 
line 409, in create_ssh_tunnel
     tunnel_password = decrypt(tunnel_password, user.password)
   File "/usr/lib/pgadmin4/web/pgadmin/utils/crypto.py", line 54, in decrypt
     cipher = AES.new(pad(key), AES.MODE_CFB, iv)
   File "/usr/lib/python3.7/site-packages/Crypto/Cipher/AES.py", line 
232, in new
     return _create_cipher(sys.modules[__name__], key, mode, *args, 
**kwargs)
   File "/usr/lib/python3.7/site-packages/Crypto/Cipher/__init__.py", 
line 79, in _create_cipher
     return modes[mode](factory, **kwargs)
   File "/usr/lib/python3.7/site-packages/Crypto/Cipher/_mode_cfb.py", 
line 270, in _create_cfb_cipher
     cipher_state = factory._create_base_cipher(kwargs)
   File "/usr/lib/python3.7/site-packages/Crypto/Cipher/AES.py", line 
103, in _create_base_cipher
     result = start_operation(c_uint8_ptr(key),
   File "/usr/lib/python3.7/site-packages/Crypto/Util/_raw_api.py", line 
144, in c_uint8_ptr
     raise TypeError("Object type %s cannot be passed to C code" % 
type(data))
TypeError: Object type <class 'str'> cannot be passed to C code
2019-01-03 15:36:23,041: ERROR    flask.app:    Could not connected to 
server(#2) - 'smr_imgur_stuff'.
Error: Failed to decrypt the SSH tunnel password.
Error: Object type <class 'str'> cannot be passed to C code
2019-01-03 15:40:04,070: ERROR    flask.app:    Object type <class 
'str'> cannot be passed to C code

After encoding the key (if it is a string) in 
pgadmin4/web/pgadmin/utils/crypto.py everything works again like expected.


--- crypto.py    2019-01-03 15:59:41.524945284 +0100

+++ /usr/lib/pgadmin4/web/pgadmin/utils/crypto.py    2019-01-03 
17:02:40.394875557 +0100
@@ -39,30 +39,32 @@


  def decrypt(ciphertext, key):
      """
      Decrypt the AES encrypted string.

      Parameters:
          ciphertext -- Encrypted string with AES method.
          key        -- key to decrypt the encrypted string.
      """

      global padding_string

      ciphertext = base64.b64decode(ciphertext)
      iv = ciphertext[:AES.block_size]
+    if hasattr(key, 'encode'):
+        key = key.encode('utf-8')
      cipher = AES.new(pad(key), AES.MODE_CFB, iv)
      decrypted = cipher.decrypt(ciphertext[AES.block_size:])

      return decrypted


Python version is: 3.7.1

If more information is required please let me know.

Thanks,

- SleepProgger