This repository has served its purpose and is no longer used. Instead, refer to the production specification at waltti-apc-vehicle-specification.
This repository contains the technical specifications for the automatic passenger counting (APC) pilot by Waltti. The text is written for the pilot partners, i.e. the vendors who create the onboard counting devices.
The devices are expected to count the amount of people and later other objects in the vehicles.
Each counting system, i.e. each set of devices of a pilot partner installed in each vehicle, is expected to send its results as soon as possible or at each stop.
The delivery out of the vehicle must happen over MQTT. Below are the MQTT and message specifications.
Each message is in JSON.
apc-from-vehicle.schema.json contains the preliminary JSON Schema that validates the data serialization for the pilot. All counting systems must conform to this schema.
Run
./validate-schema.shto validate the schema.
preferred-per-door-example.json contains a JSON example of what should be sent to the MQTT broker. All pilot partners are strongly encouraged to send data according to this example.
Run
./validate-preferred-per-door-example.shto validate the preferred example. You can modify the script for your needs.
Each counting system has its own topic:
apc-from-vehicle/<api-version>/fi/waltti/<partner-id>/<counting-system-id>
Replace <api-version> with v1.
Replace <partner-id> with one of these:
deal-compemblicaoem-finlandtelia
Replace <counting-system-id> with the countingSystemId described in apc-from-vehicle.schema.json.
We might move the MQTT brokers (servers) so do not hardcode the IP addresses.
The MQTT broker for the development environment is at dev.mqtt.apc.lmj.fi.
You can use the development environment for your research and development of the counting system.
We allow you both to publish and to subscribe onto your messages in the development environment.
However, in the production environment you will only be able to publish messages.
The pilot phase of the project uses the development environment.
Currently we are using the MQTT protocol version 3.1.1 Plus Errata 01.
Use port 8883 with TLS.
Authenticate with a username and a password.
You will be allowed to act within the topic tree:
apc-from-vehicle/v1/fi/waltti/<partner-id>/#
where <partner-id> is described in the section MQTT topic.
Reconnect automatically.
This is usually handled by the MQTT client library in the client configuration or in the connectionLostCallback() or equivalent.
Use QoS 1.
This is usually defined in the arguments for publish() or equivalent of the MQTT client library.
Do not use retained messages.
This is usually defined in the arguments for publish() or equivalent of the MQTT client library.
Set clean session to false, i.e. do not wipe out in-progress message acknowledgments upon reconnecting.
This is usually defined in the arguments for connect() or equivalent of the MQTT client library.
The Client Identifier (ClientId) names the MQTT session that persists over several TCP connections when "clean session" is set to false. For example MQTT message acknowledgment is based on ClientId.
Unfortunately the MQTT protocol version 3.1.1 has a protocol bug enabling a denial-of-service attack on connected clients: If any two clients, no matter their credentials, connect with the same ClientId, the broker must disconnect the first connection.
The MQTT brokers used in the pilot might be used by consumer-facing services in the future. Therefore the ClientId must be protected.
For each MQTT client/vehicle use a ClientId of the form:
<partner-id>-<random-suffix>
where <partner-id> is described in the section MQTT topic and <random-suffix> is 10 characters from the range [0-9A-Za-z] randomly generated by you for each counting system.
Normally there is no need to change <random-suffix>.
Do not change <random-suffix> unless you are sure all messages from the counting system have been delivered and every message has been fully acknowledged.
Treat <random-suffix> as a sensitive secret like a password.
MQTT protocol version 5.0 solves this bug but we do not support it, yet.
OPTIONAL: Using Last Will is an easy and cheap investment into your connection debugging pleasures.
Use a connection status topic:
apc-from-vehicle/<api-version>/fi/waltti/<partner-id>/<counting-system-id>/connection-status
The section on the MQTT topic explains the topic variables.
When connecting to the broker with the MQTT CONNECT message, specify a Last Will UTF-8 message payload disconnected into the connection status topic.
Ask broker to send the Last Will as a retained message with QoS 2.
These are usually defined in the arguments for connect() or equivalent of the MQTT client library.
When connecting or reconnecting, first send a UTF-8 message connected at <connected-time> to the connection status topic.
<connected-time> should be created as early as possible when the MQTT (re)connection succeeds.
It must contain an ISO 8601 timestamp with a fixed precision, for example millisecond precision, and with UTC time zone denoted with Z, e.g. 20220127T082928.123Z.
This message should be sent as a retained message with QoS 2, too.
This is usually handled in the connectionSucceededCallback() or equivalent of the MQTT client library.
OPTIONAL: To avoid data loss due to power outages or connection problems, we recommend persisting messages on non-volatile storage until acknowledged by the MQTT broker.
You could for example retain all messages in a deque or ring buffer that can contain messages for a week. Another data structure may be used instead as long as the oldest messages are thrown away first if a size limit is reached.
MQTT client libraries will not usually do this for you but will offer the necessary callbacks.