Script Transformation

Question and objective

How do you adapt a raw event whose structure is not yet suitable for analysis? A small transformation can, for example, extract a numeric value from a text field or standardize a timestamp.

Implementation in Bytefabrik

The StreamPipes connectivity transformation editor processes sample data before the final schema is defined. For JavaScript, it uses transform(event, out, ctx). Results are emitted with out.collect(...); the preview helps compare input and output. Available languages depend on the installation.

This example expects timestamp_ms to contain a numeric event time in milliseconds and temperature_raw to contain a number or a non-empty numeric string:

function transform(event, out, ctx) {
  const raw = event.temperature_raw;
  if (raw === null || raw === undefined || String(raw).trim() === '') {
    throw new Error('temperature_raw is missing');
  }
  const temperature = Number(raw);
  if (!Number.isFinite(temperature) ||
      typeof event.timestamp_ms !== 'number' ||
      !Number.isFinite(event.timestamp_ms)) {
    throw new Error('Measurement or timestamp is invalid');
  }
  out.collect({
    timestamp: event.timestamp_ms,
    temperature: temperature,
    machineId: event.machineId,
  });
}

Then mark the time field in the schema. The example makes invalid inputs explicit; the desired error behavior in the running adapter must be checked separately.

Best practices and considerations

Keep each script focused on a clearly named task. Test normal values, missing fields, and edge cases. Do not replace missing measurements with null, or null with numeric zero, without checking the implications.

A newly added current timestamp is acquisition time, not a reconstructed machine event time. Stateful operational logic belongs in an appropriate pipeline or AI Pipeline, rather than in an unwieldy connectivity script.