DataPoints - Access object for time series data (data point format)¶
Provides access API to the data point resources. Data points are resources that store time series data in a binary format.
Request Methods¶
Warning
Be sure to create the access object using data_points on client.
client = intdash.Client(...)
client.data_points.list(edge_name='test_edge', start='2020/07/01 09:00+09:00', end='2020/07/01 10:00+09:00')
- class DataPoints(...)[source]¶
Access object for time series data (data point resource).
- list(start, end, measurement_uuid=None, edge_uuid=None, edge_name=None, id_queries=None, labels=None, limit=None, iterator=False, exit_on_error=False)[source]¶
Retrieves a list of time series data (data points).
- Parameters
start (pandas.Timestamp) – Start point of retrieval range
end (pandas.Timestamp) – End point of retrieval range
measurement_uuid (str) – UUID of the source measurement
edge_uuid (str) – UUID of the source edge
edge_name (str) – Name of the source edge
id_queries (list[IdQuery]) – List of IDs of the data to be retrieved
labels (list[str]) – List of labels of the data to be retrieved
limit (int) – Maximum number of items retrieved
iterator (bool) – If True, create an iterator.
exit_on_error (bool) – If true, if an error occurs during the retrieval, the process is aborted and the list of DataResponses up to the point of interruption is returned.
- Returns
Data object
- Return type
list[DataResponse]
Examples
data_payload of the output intdash.DataResponse is binary data. You can get the physical value by using from_payload() of intdash.data.Data according to the data_type.
>>> dps = client.data_points.list( measurement_uuid=sample_measurement.uuid, id_queries=[ intdash.IdQuery( data_type=intdash.DataType.int.value, data_id='test' ) ], start=timeutils.str2timestamp("2020-01-01T00:00:00.000000Z"), end=timeutils.str2timestamp("2020-01-02T00:00:00.000000Z"), )
>>> print(intdash.data.Int.from_payload(dps[0].data_payload)) data_type: int data_id: test value: 2
Note
Specify one of
measurement_uuid
,edge_uuid
, oredge_name
. If specified at the same time, they will be referenced with the priority ofmeasurement_uuid
>edge_uuid
>edge_name
. Low priority parameters will be ignored.Note
If both
labels
andid_queries
are specified, all data that matches either of them will be retrieved. (When usinglabels
, it is necessary to register a signal definition separately.)Note
If
limit
is not specified, all data in the specified range will be retrieved.If the size of the data to be retrieved is large, you can use the iterator that retrieves the data for each limit by specifying the upper limit for the number of retrievals inlimit
and settingiterator
toTrue
.For more information, see Retrieving time series data in Tutorial.Note
When an exception occurs in the retrieval process on the server side, the DataResponse that stores the exception message is output and the process ends normally (see the example below). The
data_type
of this Unit is String. If you want to interrupt the process when an exception occurs, assignTrue
toexit_on_error
.Examples
The following is an example when an exception occurs during data retrieval. The error message is stored in a DataResponse. The data_type of the DataResponse that stores the error message is String. The namespace of the endpoint where the error occurred is output to id. You can use the information in the DataResponse to retry or to investigate the cause.
>>> dps = lc.data_points.list( measurement_uuid=sample_measurement.uuid, labels=['nmea'], start=sample_measurement.basetime, end=sample_measurement.basetime + pd.Timedelta(seconds=10) )
>>> import json >>> for p in dps: if 'error' in p.data_id: print(p) data = intdash.data.String.from_payload(p.data_payload) error_message = json.loads(data.value)['error_description'] raise ValueError(f'contains failed data: {error_message}') # time: 2020-06-23T05:08:25.676942+00:00 # measurement_uuid: 3b5b9bed-d509-4198-aea0-54ee714f7a5b # data_type: 10 # channel: 1 # data_id: intdash/measurement/get/data/error # data_payload: b'error{"error":"converted_error","error_description":"Error occurred in signal conversion","error_extra":{"signal_channel":1,"signal_data_id":"GPRMC","signal_data_type":2,"signal_label":"nmea"}}' ValueError: contains failed data: Error occurred in signal conversion
- store(measurement_uuid, data_points, serial_number=0, final=True)[source]¶
Stores time series data (data points) to the server.
- Parameters
measurement_uuid (str) – UUID of the measurement
data_points (list[DataPoint]) – List of data point objects to be stored
serial_number (int) – Section serial number
final (bool) – Final flag
Examples
Specify binary data in data_payload of DataPoint. You can create binary data with to_payload() of intdash.data.Data that matches the data_type.
>>> data_points = [ DataPoint( elapsed_time=pd.Timedelta(seconds=0) , data_type=DataType.int.value, channel=1, data_payload=intdash.data.Int(data_id='test', value=2).to_payload() ) ]
>>> client.data_points.store( measurement_uuid=sample_measurement.uuid, data_points=data_points )
Note
The maximum amount of
units
that can be specified in a single operation is 2GB. if it exceeds 2GB, runstore()
multiple times.
Request Params¶
- class DataPoint(...)[source]¶
An object that represents a DataPoint resource. Used when storing data to the server.
- elapsed_time¶
Elapsed time of data
- Type
pandas.Timedelta
- channel¶
Channel number
- Type
int
- data_payload¶
Data payload
- Type
bytes
Response¶
Refer to Time series data objects for the data types.
- class DataResponse(...)[source]¶
The data points retrieved from the server are represented as a DataResponse. The payload can be obtained with data_payload.
- time¶
Data start time
- Type
pandas.Timestamp
- measurement_uuid¶
Measurement UUID
- Type
str
- data_id¶
Data ID
- Type
str
- channel¶
Channel number
- Type
int
- data_payload¶
Data payload
- Type
bytes