Define JSON object with template information to hold version data

[TonyWildish-BH]

(This ticket has been broken into several smaller pieces since it was originally written).

The Base workspace template is defined by a JSON object. This needs to be extended to hold information about all the workspace services and their user resources, specifically to list which versions are available to the workspace. The goal for this ticket is to be able to create that extended information.

The JSON format for the extension will look something like this:

[
 {
  "name": "tre-service-guacamole",
  "title": "Apache Guacamole - Virtual Desktop Service",
  "versions": [
   {
    "version": "0.12.6",
    "description": "Enables Windows and Linux virtual machines to be accessed via Apache Guacamole.",
    "enabled": {
     "TRE": false,
     "Workspace": false
    }
   },
 {
    "version": "1.20.2",
    "description": "Enables Windows and Linux virtual machines to be accessed via Apache Guacamole.",
    "enabled": {
     "TRE": true,
     "Workspace": true
    }
  ],
  "user-resources": [
   {
    "name": "tre-service-guacamole-linuxvm",
    "versions": [
     {
      "version": "1.2.3",
      "description": "Linux virtual machine.",
      "enabled": {
       "TRE": false,
       "Workspace": false
      }
     }
    ]
   },
   {
    "name": "tre-service-guacamole-windowsvm",
    "versions": [
     {
        ...   
     } 
  ]
},
{
  "name": "tre-service-databricks",
  "title": "Azure Databricks",
  "versions": [
    ....
  ],
  ....
 }
]

The essential points are:

• it has the list of all workspace services
• for each service, it lists the versions
• for each service, list the child resources (user-resources) recursively
• when creating the data structure for the first time, all version keys are set to "False".

The "versions" sub-structure should have entries in order. String-sorting may not work, since the version number has MAJOR.MINOR.PATCH structure. This will help with presentation in the UI, the fields will already be sorted.

The information to fill this structure comes from the list of registered templates in CosmosDB.

Acceptance criteria:

• A script that trawls the CosmosDB to find the list of template versions, and creates the data structure specified above. The script can be Python or Bash, using either a native Azure/Cosmos CLI or a REST API. The structure should be written to a local file as pure JSON.
• Specification of any firewall changes or IAM roles needed to allow a user to run this script from an external location.

[TonyWildish-BH]

N.B., eventually, we will need two boolean flags per version, one for the TRE Admin and one for the Workspace Admin. The Workspace Admin will be able to change their flag only if the TRE Admin has enabled that version for a workspace (see the Service Catalogue slide deck we discussed recently).

This suggests that the version array should look something like this, instead:

      "versions": [
        { "1.0.0": [ False, False ] },
        { "1.2.0": [ True, True ] },
        { "2.0.0": [ True, False ] },
      ],

where, for example, the TRE Admin manipulates the first element, and the Workspace Admin manipulates the second element. So in this example, version 1.0.0 is disabled, and only the TRE Admin can enable it. Version 1.2.0 is enabled, and either role can enable/disable it. Version 2.0.0 is allowed by the TRE Admin, but the Workspace Admin has not enabled it for their project.

[TonyWildish-BH]

To clarify from the Show & Tell, I envision this object being built in the workspace CosmosDB entry when the workspace is created, by trawling the list of ResourceTemplates that are available from CosmosDB.

The object should be updated (refreshed) with new versions when the TRE Admin goes into the Configuration tab to configure the set of versions available for each workspace. That's covered in #191, and doesn't need to be done yet.

The workflow we discussed, as I understand it, and I'm happy to be corrected, would look like this:

• Monday: TRE Admin creates Workspace_A. Versions object in workspace object is populated with existing resource templates at that time. So, for example, this includes AzureML version 1.0.0.
• Tuesday: TRE Admin creates new version of a template, e.g. AzureML template version 2.0.0. No workspace objects are updated at this point.
• Wednesday: Workspace_A Admin comes along and deploys AzureML. They only have version 1.0.0 available to them.
• Thursday: TRE Admin goes into Workspace_A and opens the Configuration tab. At that point, the ResourceTemplate list is scanned again, and is merged into the workspace. AzureML 2.0.0 then shows up, disabled by default. The TRE Admin can enable it, and save the configuration.
• Friday: Workspace_A Admin comes along and deploys another AzureML (maybe they now have two, maybe they deleted the first one, no matter). This time, they get version 2.0.0 by default (latest active version in their workspace).

Future enhancements will allow them to choose between version 1.0.0 and 2.0.0 if both are still enabled, but for now, it's enough to deploy the latest active version.

Another possible enhancement is for when we start to allow the Workspace Admin to control versions themselves, and that's to do the object refresh when the Workspace Admin goes into the Configuration tab. That's out of scope for this epic, we're only working with the TRE Admin access.

[EDIT: This gives the big picture, but this ticket has been down-scoped since I added this bit. Updating the workspace will happen elsewhere.]

[E-PAC]

MR
https://github.com/Barts-Life-Science/AzureTRE/pull/new/feature/template_version_number_misc

USAGE

1. run script
2. create object in cosmosDB
3. edit object with script output

[alan-braggins-pa]

From Teams chat: "all the workspace services" means objects with resourceType "workspace-service" (not "workspace" nor "shared-service"), "their user resources" means resourceType "user-resource" and parentWorkspaceService matches the name of the owning workspace-service
Template items with different version will be matched on name (e.g. "tre-service-guacamole"), and the title and description (e.g. "Apache Guacamole - Virtual Desktop Service", "Enables Windows and Linux virtual machines to be accessed via Apache Guacamole.") should be included.
The descriptions could, in principle, be different in different versions of the same template. If they are different, including the description for each version might be helpful for the admin who will edit the file for ticket #217 to apply changes.

[alan-braggins-pa]

Updated JSON example structure to match discussion in Teams. Descriptions and title included for editor context, static key names to support query like
cat a.json | jq -c '.[] | select( .name == "tre-service-guacamole" ) | .versions[] | select( .enabled.TRE == true ) | .version'

[alan-braggins-pa]

IAM roles - I added id-api-abmstre1 as a User Assigned Managed Identity to my VM, assuming that this would end up integrated with the API eventually, although it has a lot more permissions than needed just for this script.
Set AZURE_CLIENT_ID in the environment or use the --client_id argument (which then sets it in the environment for DefaultAzureCredential() to pick up).

I assigned "Cosmos DB Built-in Data Contributor" to myself as a user to enable queries in the Portal Data Explorer.
I suspect that would be sufficient for the script, but haven't tested it it.
https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/security/how-to-grant-data-plane-role-based-access?tabs=built-in-definition%2Ccsharp&pivots=azure-interface-cli#assign-role-to-identity

az cosmosdb sql role definition list --resource-group rg-abmstre1 --account-name cosmos-abmstre1
az cosmosdb show --resource-group rg-abmstre1 --name cosmos-abmstre1 --query {id:id}
az cosmosdb sql role assignment create --resource-group rg-abmstre1 --account-name cosmos-abmstre1 --role-definition-id "/subscriptions/39810b76-7b5b-4984-ba0b-32f5376634e6/resourceGroups/rg-abmstre1/providers/Microsoft.DocumentDB/databaseAccounts/cosmos-abmstre1/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002" --principal-id "9d6f3297-af5f-4cc9-8524-c8383edc6ecd" --scope "/subscriptions/39810b76-7b5b-4984-ba0b-32f5376634e6/resourceGroups/rg-abmstre1/providers/Microsoft.DocumentDB/databaseAccounts/cosmos-abmstre1"

(Where --principal-id "9d6f3297-af5f-4cc9-8524-c8383edc6ecd" is from looking up myself on
https://entra.microsoft.com/#view/Microsoft_AAD_UsersAndTenants/UserManagementMenuBlade/~/AllUsers/menuId/)

[alan-braggins-pa]

The CosmosDB network settings I allowed access from the IP address of my VM, and for the corporate IP ranges my laptop appeared in for the Portal CosmosDB Data Explorer. (There is a group listed on the CosmosDB controls for "Portal Middleware", but Data Explorer was blocked as coming from my laptop address, so wasn't using the middleware.)
(I also allowed ssh access to my VM from the same IP ranges).