DataPoints - Access object for time series data (data point format)

Provides access API to the data point resources. Data points are resources that store time series data in a binary format.

Request Methods

Warning

Be sure to create the access object using data_points on client.

client = intdash.Client(...)
client.data_points.list(edge_name='test_edge', start='2020/07/01 09:00+09:00', end='2020/07/01 10:00+09:00')
class DataPoints(...)[source]

Access object for time series data (data point resource).

list(start, end, measurement_uuid=None, edge_uuid=None, edge_name=None, id_queries=None, labels=None, limit=None, iterator=False, exit_on_error=False)[source]

Retrieves a list of time series data (data points).

Parameters
  • start (pandas.Timestamp) – Start point of retrieval range

  • end (pandas.Timestamp) – End point of retrieval range

  • measurement_uuid (str) – UUID of the source measurement

  • edge_uuid (str) – UUID of the source edge

  • edge_name (str) – Name of the source edge

  • id_queries (list[IdQuery]) – List of IDs of the data to be retrieved

  • labels (list[str]) – List of labels of the data to be retrieved

  • limit (int) – Maximum number of items retrieved

  • iterator (bool) – If True, create an iterator.

  • exit_on_error (bool) – If true, if an error occurs during the retrieval, the process is aborted and the list of DataResponses up to the point of interruption is returned.

Returns

Data object

Return type

list[DataResponse]

Examples

data_payload of the output intdash.DataResponse is binary data. You can get the physical value by using from_payload() of intdash.data.Data according to the data_type.

>>> dps = client.data_points.list(
        measurement_uuid=sample_measurement.uuid,
        id_queries=[
            intdash.IdQuery(
                data_type=intdash.DataType.int.value,
                data_id='test'
            )
        ],
        start=timeutils.str2timestamp("2020-01-01T00:00:00.000000Z"),
        end=timeutils.str2timestamp("2020-01-02T00:00:00.000000Z"),
    )
>>> print(intdash.data.Int.from_payload(dps[0].data_payload))
data_type: int
data_id: test
value: 2

Note

Specify one of measurement_uuid, edge_uuid, or edge_name. If specified at the same time, they will be referenced with the priority of measurement_uuid > edge_uuid > edge_name. Low priority parameters will be ignored.

Note

If both labels and id_queries are specified, all data that matches either of them will be retrieved. (When using labels, it is necessary to register a signal definition separately.)

Note

If limit is not specified, all data in the specified range will be retrieved.If the size of the data to be retrieved is large, you can use the iterator that retrieves the data for each limit by specifying the upper limit for the number of retrievals in limit and setting iterator to True.For more information, see Retrieving time series data in Tutorial.

Note

When an exception occurs in the retrieval process on the server side, the DataResponse that stores the exception message is output and the process ends normally (see the example below). The data_type of this Unit is String. If you want to interrupt the process when an exception occurs, assign True to exit_on_error.

Examples

The following is an example when an exception occurs during data retrieval. The error message is stored in a DataResponse. The data_type of the DataResponse that stores the error message is String. The namespace of the endpoint where the error occurred is output to id. You can use the information in the DataResponse to retry or to investigate the cause.

>>> dps = lc.data_points.list(
            measurement_uuid=sample_measurement.uuid,
            labels=['nmea'],
            start=sample_measurement.basetime,
            end=sample_measurement.basetime + pd.Timedelta(seconds=10)
        )
>>> import json
>>> for p in dps:
        if 'error' in p.data_id:
            print(p)
            data = intdash.data.String.from_payload(p.data_payload)
            error_message = json.loads(data.value)['error_description']
            raise ValueError(f'contains failed data: {error_message}')
# time: 2020-06-23T05:08:25.676942+00:00
# measurement_uuid: 3b5b9bed-d509-4198-aea0-54ee714f7a5b
# data_type: 10
# channel: 1
# data_id: intdash/measurement/get/data/error
# data_payload: b'error{"error":"converted_error","error_description":"Error occurred in signal conversion","error_extra":{"signal_channel":1,"signal_data_id":"GPRMC","signal_data_type":2,"signal_label":"nmea"}}'
ValueError: contains failed data: Error occurred in signal conversion
store(measurement_uuid, data_points, serial_number=0, final=True)[source]

Stores time series data (data points) to the server.

Parameters
  • measurement_uuid (str) – UUID of the measurement

  • data_points (list[DataPoint]) – List of data point objects to be stored

  • serial_number (int) – Section serial number

  • final (bool) – Final flag

Examples

Specify binary data in data_payload of DataPoint. You can create binary data with to_payload() of intdash.data.Data that matches the data_type.

>>> data_points = [
        DataPoint(
            elapsed_time=pd.Timedelta(seconds=0) ,
            data_type=DataType.int.value,
            channel=1,
            data_payload=intdash.data.Int(data_id='test', value=2).to_payload()
        )
    ]
>>> client.data_points.store(
        measurement_uuid=sample_measurement.uuid,
        data_points=data_points
    )

Note

The maximum amount of units that can be specified in a single operation is 2GB. if it exceeds 2GB, run store() multiple times.

Request Params

class DataPoint(...)[source]

An object that represents a DataPoint resource. Used when storing data to the server.

elapsed_time

Elapsed time of data

Type

pandas.Timedelta

channel

Channel number

Type

int

data_type

Data type

Type

DataType

data_payload

Data payload

Type

bytes

class IdQuery(...)[source]

An object that represents the condition definition of data.

data_type

Data type

Type

DataType

channel

Channel

Type

int

data_id

Data ID

Type

str

class DataType(...)[source]

A constant that represents the data type.

aac = 16
basetime = 135
bytes = 14
can = 1
can_bulk = 7
controlpad = 4
float = 11
general_sensor = 3
general_sensor_bulk = 8
generic = 127
h264 = 13
int = 12
jpeg = 9
nmea = 2
pcm = 15
string = 10

Response

Refer to Time series data objects for the data types.

class DataResponse(...)[source]

The data points retrieved from the server are represented as a DataResponse. The payload can be obtained with data_payload.

time

Data start time

Type

pandas.Timestamp

measurement_uuid

Measurement UUID

Type

str

data_type

Data type

Type

DataType

data_id

Data ID

Type

str

channel

Channel number

Type

int

data_payload

Data payload

Type

bytes