Derive JSON Schema files from YAML files

This lets us add detailed documentation & description
to our schemas, which is very hard to do in JSON.

We also add a lot of documentation to the one
JSON schema we have

jupyter_server/event-schemas/README.md

@@ -0,0 +1,19 @@
+# Event Schemas
+
+## Generating .json files
+
+Event Schemas are written in a human readable `.yaml` format.
+This is primarily to get multi-line strings in our descriptions,
+as documentation is very important.
+
+Every time you modify a `.yaml` file, you should run the following
+commands.
+
+```bash
+./generate-json.py
+```
+
+This needs the `ruamel.yaml` python package installed.
+
+Hopefully, this is extremely temporary, and we can just use YAML
+with jupyter_telemetry.

jupyter_server/event-schemas/contentsmanager-actions.json

@@ -2,9 +2,12 @@
     "$id": "eventlogging.jupyter.org/notebook/contentsmanager-actions",
     "version": 1,
     "title": "Contents Manager activities",
-    "description": "Notebook Server emits this event whenever a contentsmanager action happens",
+    "description": "Record actions on files via the ContentsManager REST API.\n\nThe notebook ContentsManager REST API is used by all frontends to retreive,\nsave, list, delete and perform other actions on notebooks, directories,\nand other files through the UI. This is pluggable - the default acts on\nthe file system, but can be replaced with a different ContentsManager\nimplementation - to work on S3, Postgres, other object stores, etc.\nThe events get recorded regardless of the ContentsManager implementation\nbeing used.\n\nLimitations:\n\n1. This does not record all filesystem access, just the ones that happen\n   explicitly via the notebook server's REST API. Users can (and often do)\n   trivially access the filesystem in many other ways (such as `open()` calls\n   in their code), so this is usually never a complete record.\n2. As with all events recorded by the notebook server, users most likely\n   have the ability to modify the code of the notebook server. Unless other\n   security measures are in place, these events should be treated as user\n   controlled and not used in high security areas.\n3. Events are only recorded when an action succeeds.\n",
     "type": "object",
-    "required": ["action", "path"],
+    "required": [
+        "action",
+        "path"
+    ],
     "properties": {
         "action": {
             "enum": [
@@ -13,18 +16,18 @@
                 "save",
                 "upload",
                 "rename",
-                "create",
-                "copy"
+                "copy",
+                "delete"
             ],
-            "description": "Action performed by contents manager"
+            "description": "Action performed by the ContentsManager API.\n\nThis is a required field.\n\nPossible values:\n\n1. get\n   Get contents of a particular file, or list contents of a directory.\n\n2. create\n   Create a new directory or file at 'path'. Currently, name of the\n   file or directory is auto generated by the ContentsManager implementation.\n\n3. save\n   Save a file at path with contents from the client\n\n4. upload\n   Upload a file at given path with contents from the client\n\n5. rename\n   Rename a file or directory from value in source_path to\n   value in path.\n\n5. copy\n   Copy a file or directory from value in source_path to\n   value in path.\n\n6. delete\n   Delete a file or empty directory at given path\n"
         },
         "path": {
             "type": "string",
-            "description": "Logical path the action was performed in"
+            "description": "Logical path on which the operation was performed.\n\nThis is a required field.\n"
         },
         "source_path": {
             "type": "string",
-            "description": "If action is 'copy', this specifies the source path"
+            "description": "Source path of an operation when action is 'copy' or 'rename'"
         }
     }
 }

jupyter_server/event-schemas/generate-json.py

@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import os
+import jsonschema
+from ruamel.yaml import YAML
+
+from jupyter_telemetry.eventlog import EventLog
+
+yaml = YAML(typ='safe')
+
+def main():
+    argparser = argparse.ArgumentParser()
+    argparser.add_argument(
+        'directory',
+        help='Directory with Schema .yaml files'
+    )
+
+    args = argparser.parse_args()
+
+    el = EventLog()
+    for dirname, _, files in os.walk(args.directory):
+        for file in files:
+            if not file.endswith('.yaml'):
+                continue
+            yaml_path = os.path.join(dirname, file)
+            print('Processing', yaml_path)
+            with open(yaml_path) as f:
+                schema = yaml.load(f)
+
+            # validate schema
+            el.register_schema(schema)
+
+            json_path = os.path.join(dirname, os.path.splitext(file)[0] + '.json')
+            with open(json_path, 'w') as f:
+                json.dump(schema, f, indent=4)
+
+if __name__ == '__main__':
+    main()

jupyter_server/event-schemas/v1.yaml

@@ -0,0 +1,79 @@
+"$id": eventlogging.jupyter.org/notebook/contentsmanager-actions
+version: 1
+title: Contents Manager activities
+description: |
+    Record actions on files via the ContentsManager REST API.
+
+    The notebook ContentsManager REST API is used by all frontends to retreive,
+    save, list, delete and perform other actions on notebooks, directories,
+    and other files through the UI. This is pluggable - the default acts on
+    the file system, but can be replaced with a different ContentsManager
+    implementation - to work on S3, Postgres, other object stores, etc.
+    The events get recorded regardless of the ContentsManager implementation
+    being used.
+
+    Limitations:
+
+    1. This does not record all filesystem access, just the ones that happen
+       explicitly via the notebook server's REST API. Users can (and often do)
+       trivially access the filesystem in many other ways (such as `open()` calls
+       in their code), so this is usually never a complete record.
+    2. As with all events recorded by the notebook server, users most likely
+       have the ability to modify the code of the notebook server. Unless other
+       security measures are in place, these events should be treated as user
+       controlled and not used in high security areas.
+    3. Events are only recorded when an action succeeds.
+type: object
+required:
+- action
+- path
+properties:
+  action:
+    enum:
+    - get
+    - create
+    - save
+    - upload
+    - rename
+    - copy
+    - delete
+    description: |
+        Action performed by the ContentsManager API.
+
+        This is a required field.
+
+        Possible values:
+
+        1. get
+           Get contents of a particular file, or list contents of a directory.
+
+        2. create
+           Create a new directory or file at 'path'. Currently, name of the
+           file or directory is auto generated by the ContentsManager implementation.
+
+        3. save
+           Save a file at path with contents from the client
+
+        4. upload
+           Upload a file at given path with contents from the client
+
+        5. rename
+           Rename a file or directory from value in source_path to
+           value in path.
+
+        5. copy
+           Copy a file or directory from value in source_path to
+           value in path.
+
+        6. delete
+           Delete a file or empty directory at given path
+  path:
+    type: string
+    description: |
+        Logical path on which the operation was performed.
+
+        This is a required field.
+  source_path:
+    type: string
+    description: |
+        Source path of an operation when action is 'copy' or 'rename'