Output of JSON simulation_log files use an incorrect format

[Lance-Drane]

Running the sample helloworld usecase, this is an example of one of the files which is outputted in simulation_log:

helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496.json

{"code": "Framework", "eventtype": "IPS_START", "ok": true, "comment": "Starting IPS Simulation", "walltime": "0.10", "time": "2023-11-03|14:13:48EDT", "state": "Running", "host": "crissaegrim", "simname": "helloworld", "user": "ld0", "rcomment": "SWIM simulation run", "startat": "2023-11-03|14:13:48EDT", "ips_version": "0.8.1", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "parent_portal_runid": null, "seqnum": 0}
{"code": "Framework", "eventtype": "IPS_RESOURCE_ALLOC", "ok": true, "comment": "Nodes = 1   PPN = 1", "walltime": "0.10", "time": "2023-11-03|14:13:48EDT", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 1}
{"code": "Framework", "eventtype": "IPS_CALL_BEGIN", "ok": true, "comment": "Target = helloworld@hello_driver@1:init(0)", "walltime": "0.10", "time": "2023-11-03|14:13:48EDT", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 2}
{"code": "Framework", "eventtype": "IPS_CALL_END", "ok": true, "comment": "Target = helloworld@hello_driver@1:init(0)", "walltime": "0.10", "time": "2023-11-03|14:13:48EDT", "trace": {"timestamp": 1699035228485604, "duration": 1293, "localEndpoint": {"serviceName": "helloworld@hello_driver@1"}, "name": "init(0)", "id": "6b53f810a36f143c", "parentId": "71096617153e23c3", "traceId": "617350e2798afdda7e0bd6257faf1ff9"}, "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 3}
{"code": "Framework", "eventtype": "IPS_CALL_BEGIN", "ok": true, "comment": "Target = helloworld@hello_driver@1:step(0)", "walltime": "0.10", "time": "2023-11-03|14:13:48EDT", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 4}
{"code": "driver__hello_driver", "eventtype": "IPS_CALL_BEGIN", "ok": true, "walltime": "0.11", "time": "2023-11-03|14:13:48EDT", "state": "Running", "comment": "Target = helloworld@hello_worker@2:step(0.000)", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 5}
{"code": "driver__hello_driver", "eventtype": "IPS_CALL_END", "ok": true, "walltime": "0.16", "time": "2023-11-03|14:13:48EDT", "trace": {"timestamp": 1699035228533220, "duration": 11200, "localEndpoint": {"serviceName": "helloworld@hello_worker@2"}, "name": "step(0.000)", "id": "0e5bef2d18930f46", "parentId": "72dba9575cc71d4a", "tags": {}, "traceId": "617350e2798afdda7e0bd6257faf1ff9"}, "state": "Running", "comment": "Target = helloworld@hello_worker@2:step(0.000)", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 6}
{"code": "Framework", "eventtype": "IPS_CALL_END", "ok": true, "comment": "Target = helloworld@hello_driver@1:step(0)", "walltime": "0.18", "time": "2023-11-03|14:13:48EDT", "trace": {"timestamp": 1699035228487163, "duration": 80933, "localEndpoint": {"serviceName": "helloworld@hello_driver@1"}, "name": "step(0)", "id": "72dba9575cc71d4a", "parentId": "71096617153e23c3", "traceId": "617350e2798afdda7e0bd6257faf1ff9"}, "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 7}
{"code": "Framework", "eventtype": "IPS_CALL_BEGIN", "ok": true, "comment": "Target = helloworld@hello_driver@1:finalize(0)", "walltime": "0.18", "time": "2023-11-03|14:13:48EDT", "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 8}
{"code": "Framework", "eventtype": "IPS_CALL_END", "ok": true, "comment": "Target = helloworld@hello_driver@1:finalize(0)", "walltime": "0.18", "time": "2023-11-03|14:13:48EDT", "trace": {"timestamp": 1699035228568647, "duration": 974, "localEndpoint": {"serviceName": "helloworld@hello_driver@1"}, "name": "finalize(0)", "id": "669ed0ad2e13ce6c", "parentId": "71096617153e23c3", "traceId": "617350e2798afdda7e0bd6257faf1ff9"}, "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 9}
{"code": "Framework", "eventtype": "IPS_END", "ok": true, "comment": "Simulation Ended", "walltime": "0.18", "time": "2023-11-03|14:13:48EDT", "state": "Completed", "stopat": "2023-11-03|14:13:48EDT", "trace": {"timestamp": 1699035228388625, "duration": 181280, "localEndpoint": {"serviceName": "helloworld@FRAMEWORK@Framework@0"}, "id": "71096617153e23c3", "tags": {"total_cores": "1"}, "traceId": "617350e2798afdda7e0bd6257faf1ff9"}, "sim_name": "helloworld", "phystimestamp": -1, "portal_runid": "helloworld_crissaegrim_ld0_2023-11-03T14:13:48.496", "seqnum": 10}

Condensed, the structure of this output file looks like this:

{...}
{...}
{...}

This is technically invalid JSON, and will cause errors if a user wishes to programmatically modify this file (i.e. with jq or Python's json.load) or submit this file elsewhere (i.e. curl --data "@file.json" ...).

A suggested approach would be:

[
  {...},
  {...},
  {...}
]

The underlying data structure of the objects does not need to be changed, the objects just need to be wrapped inside an array.

[Lance-Drane]

fix is to utilize the JSONL specification and write .jsonl files instead of .json files