Geckoboard Developer Community

Python API - problem with decimal numbers


#1

I’m trying to upload a dataset with the Python API.
I’m executing a Postgres SQL query:

curs.execute(sql)
rows = curs.fetchall()
print(rows)

The query returns this:

[{‘description’: ‘Order Size 2-9’, ‘amount’: 222, ‘average’: Decimal(‘2.9’)}]

 dataset.post([rows[0]])

has an error:

TypeError: Object of type Decimal is not JSON serializable

Stacktrace:

Traceback (most recent call last):
File “/xxx/python/geckoboard/geckoboard_upload_nrt.py”, line 268, in
main()
File “/xxx/python/geckoboard/geckoboard_upload_nrt.py”, line 262, in main
dwh.run()
File “/xxx/python/geckoboard/geckoboard_upload_nrt.py”, line 189, in run
dataset.post([rows[i]])
File “/usr/local/lib/python3.7/site-packages/geckoboard/dataset.py”, line 79, in post
self.__connection.post(’/datasets/’ + self.__id + ‘/data’, body)
File “/usr/local/lib/python3.7/site-packages/geckoboard/connection.py”, line 25, in post
response = api.post(path, body, self.api_key)
File “/usr/local/lib/python3.7/site-packages/geckoboard/api.py”, line 19, in post
return requests.post(API_HOST + path, json=body, auth=(api_key, ‘’))
File “/usr/local/lib/python3.7/site-packages/requests/api.py”, line 112, in post
return request(‘post’, url, data=data, json=json, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/api.py”, line 58, in request
return session.request(method=method, url=url, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 498, in request
prep = self.prepare_request(req)
File “/usr/local/lib/python3.7/site-packages/requests/sessions.py”, line 441, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File “/usr/local/lib/python3.7/site-packages/requests/models.py”, line 312, in prepare
self.prepare_body(data, files, json)
File “/usr/local/lib/python3.7/site-packages/requests/models.py”, line 462, in prepare_body
body = complexjson.dumps(json)
File “/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/init.py”, line 231, in dumps
return _default_encoder.encode(obj)
File “/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py”, line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File “/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py”, line 257, in iterencode
return _iterencode(o, 0)
File “/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py”, line 179, in default
raise TypeError(f’Object of type {o.class.name} ’
TypeError: Object of type Decimal is not JSON serializable

Dataset defined as:

  "ds_name" : "company_order_size_nrt",
  "ds_definition": {
      "amount": {
          "type": "number",
          "name": "Amount",
          "optional": false
      },
      "average": {
          "type": "number",
          "name": "Average",
          "optional": false
      },
      "description": {
          "type": "string",
          "name": "Description",
          "optional": false
      }
  }

Is it possible to upload decimal values? If yes, how to do it via the Python API?

Thanks
Sebastian


#2

I managed to workaround this by casting the DECIMAL ‘average’ field into a VARCHAR in the query which yields:

[{‘description’: ‘Order Size 1’, ‘amount’: 272, ‘average’: ‘1.00’}

This can be uploaded to the dataset without a problem.
Still it would be interesting to know how to properly upload a DECIMAL value being returned by a Postgres query in the Python API.


#3

Just curious. Have you tried using SQL-Dataset instead?


#4

No, not yet. Just read the doc. It’s definitely worth trying!