Examples of advanced data retrieval¶

This section provides more advanced examples of data retrieval.

Retrieve JPEG type data and save it as an image file
Retrieve physical values from JSON type data using signal definitions

Retrieve JPEG type data and save it as an image file ¶

In the following example, we will use the intdash SDK for Python to retrieve JPEG image data from the intdash server and save it as an image file (.jpg) in the local environment.

Retrieving time series data¶

In this scenario, we will use the client.data_points endpoint to retrieve the time series data.

For start and end, specify the start and end times of the range to be retrieved.

In id_queries, specify the data to retrieve by using data_type and channel.

Normally, you would also specify data_id in id_queries, but for the JPEG type, data_id is always 'jpeg' and does not need to be specified.

import intdash
from intdash import timeutils

client = intdash.Client(
    url = "https://example.intdash.jp",
    edge_token = "your_token",
)

dps = client.data_points.list(
    edge_name='sdk_edge1',
    start=timeutils.str2timestamp('2020-07-17 00:00:00+09:00'), # change appropriately.
    end=timeutils.str2timestamp('2020-07-18 00:00:00+09:00'), # change appropriately.
    id_queries=[
        intdash.IdQuery(
            data_type=intdash.DataType.jpeg,
            channel=1
        )
    ]
)

Save the retrieved data as an image file¶

Save the retrieved binary data as a JPEG file. Here, as an example, we will save only the first data point of the retrieved time series data.

import pandas as pd

data = dps[0]  #  Specifies first data.

# Use the timestamp as the file name.
with open(f'./{pd.Timestamp(data.time).value}.jpg', 'wb') as fout:
    fout.write(data.data_payload)

If you want to save all data points, repeat the above.

import os
import pandas as pd

save_dir='./images'

for d in dps:

    if not os.path.isdir(save_dir):
            os.makedirs(save_dir)

    # Use the timestamps as the file names.
    with open(f'{save_dir}/{pd.Timestamp(d.time).value}.jpg', 'wb') as fout:
        fout.write(d.data_payload)

Retrieve physical values from JSON type data using signal definitions ¶

In the following example, JSON data stored in intdash as a string type is deserialized on the server side, and the numerical values contained in the JSON data are retrieved.

First, make sure that the JSON data can be obtained as a string.

import intdash
from intdash import timeutils

client = intdash.Client(
    url = "https://example.intdash.jp",
    edge_token = "your_token",
)

dps = client.data_points.list(
    edge_name='sdk_edge1',
    start=timeutils.str2timestamp('2020-07-09 00:00:00+09:00'), # change appropriately.
    end=timeutils.str2timestamp('2020-07-10 00:00:00+09:00'), # change appropriately.
    id_queries=[intdash.IdQuery(
        data_type=intdash.DataType.string,
        data_id='json_data',
        channel=1
    )]
)
print(dps[0].data_payload)

# b"\tjson_data{'sp_ACCY': 0.041493, 'sp_ACCZ': 0.217996, 'sp_ACCX': 0.048094}"

Note

In intdash, JSON data is treated as a string (intdash.DataType.string).

Register signal definitions for JSON data¶

Create signal definitions to retrieve values in a JSON string as numeric values.

Register the signal definitions to the intdash server as shown in Signal definition examples for JSON(String).

Each signal definition has a label (label). The data converted with a signal definition is obtained by specifying the label.

The labels used in this scenario are as follows:

Key in JSON	Label to be given to the data after conversion to physical values
sp_ACCX	json_ACCX
sp_ACCY	json_ACCY
sp_ACCZ	json_ACCZ

Confirm the signal definitions¶

Confirm that the signal definitions are registered.

You can get a list of signal definitions with labels starting with json by specifying label='json' as follows.

signals = client.signals.list(label='json')
for s in signals:
    print(s.label,  end=', ')

Retrieve time series data by specifying the signal definitions¶

Retrieve data points by specifying the labels of the signal definitions you want to use.