Introduce packaging, distribution and documentation for the C# driver (…

…#623) ## Usage and product changes We introduce packaging, distribution and documentation C# driver for TypeDB ([the original driver PR](#609)). It is built using the cross-platform [.NET 6 framework](https://dotnet.microsoft.com/en-us/download/dotnet/6.0). ### Usage: The driver is distributed as a series of [Nuget](https://www.nuget.org) packages. To use the driver, import the latest versions of the driver (TypeDB.Driver) and its Pinvoke runtime (TypeDB.Driver.Pinvoke) suitable for your platform. **CS project**: Here is an example from a `.csproj` for MacOS x86-64: ```xml <PackageReference Include="TypeDB.Driver" Version={VERSION} /> <PackageReference Include="TypeDB.Driver.Pinvoke.osx-x64" Version={VERSION} /> ``` If you aim to build a platform-independent package, reference all the needed runtimes (it will affect the size of your application by downloading a respective set of platform-specific dynamic libraries): ```xml <PackageReference Include="TypeDB.Driver.Pinvoke.osx-x64" Version={VERSION} /> <PackageReference Include="TypeDB.Driver.Pinvoke.linux-x64" Version={VERSION} /> <PackageReference Include="TypeDB.Driver.Pinvoke.win-x64" Version={VERSION} /> <PackageReference Include="TypeDB.Driver.Pinvoke.osx-arm64" Version={VERSION} /> ... ``` **Bazel**: 1. Import both the `TypeDB.Driver` and `TypeDB.Driver.Pinvoke` nuget packages as dependencies to your build rule. 2. For development, you can also use a [csharp bazel rule](https://github.com/bazelbuild/rules_dotnet/tree/master), passing targets `//csharp:driver-csharp` (the driver itself), `//csharp/Api:api` (exposed Api namespace), `//csharp/Common:common` (exposed Common namespace) as dependencies. **A simple usage example** (see `csharp/Test/Integration/Examples` for more): ``` using TypeDB.Driver.Api; using TypeDB.Driver.Common; public class TypeDBExample { public void SetupTypeDB() { string dbName = "access-management-db"; string serverAddr = "127.0.0.1:1729"; try { using (ITypeDBDriver driver = Drivers.CoreDriver(serverAddr)) { driver.Databases.Create(dbName); IDatabase database = driver.Databases.Get(dbName); using (ITypeDBSession session = driver.Session(dbName, SessionType.Schema)) { using (ITypeDBTransaction transaction = session.Transaction(TransactionType.Write)) { transaction.Query.Define("define person sub entity;").Resolve(); string longQuery = "define name sub attribute, value string; person owns name;"; transaction.Query.Define(longQuery).Resolve(); transaction.Commit(); } } database.Delete(); } } catch (TypeDBDriverException e) { Console.WriteLine($"Caught TypeDB Driver Exception: {e}"); } } } ``` ## Implementation **Documentation:** Documentation for C# is generated in a similar way as the C/C++ ones: `doxygen` -> `adoc`. This way, the C# parser has some extra logic of parsing the intermediate doxygen files but follows a similar (copypasted) structure. Additionally, some small bugs for both old parsers have been fixed. **Distribution:** We decided to distribute the C# driver as a series of packages to allow the end user to choose between multi-platform and compactness. This way, we have: `TypeDB.Driver.{X-X-X-version}.nupkg` - the main package with the driver, which fails in runtime if no runtime is additionally provided. `TypeDB.Driver.Pinvoke.{X-X-X-version}.{platform}.nupkg` - platform-specific runtimes (`osx-x64`, `osx-arm64`, `linux-x64`, `linux-arm64`, `win-x64`). The distribution is implemented via several bazel rules: `nuget_pack` (with a wrapper for native packages in this PR) and `nuget_push` and their respective calls for both runtime and main packages. **Tests:** Integration tests use NUnit framework as a built-in framework for C# bazel test rules. This PR presents both `test-core` and `test-cloud` as small usage examples within the integration tests. These tests are also triggered by Factory. There is also a new Deployment test, which creates a `.csproj` project and tries to fetch all the needed (and all the available runtime) packages to use them for another small C# program. This test is triggered by CircleCI.
typedb · Apr 4, 2024 · 3c70b84 · 3c70b84
1 parent 07c4d65
commit 3c70b84
Show file tree

Hide file tree

Showing 367 changed files with 14,737 additions and 1,582 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -349,6 +349,81 @@ commands:
           export DEPLOY_ARTIFACT_PASSWORD=$REPO_TYPEDB_PASSWORD
           bazel run --jobs=8 --define version=$(cat VERSION) //cpp:deploy-cpp-driver --compilation_mode=opt -- release
 
+  #########################
+  # C# deployment steps #
+  #########################
+
+  install-dotnet-mac:
+    steps:
+      - run: |
+          curl -OL "https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh"
+          chmod a+x dotnet-install.sh
+          ./dotnet-install.sh --version 6.0.125
+          ln -s /Users/distiller/.dotnet/dotnet /usr/local/bin/
+
+  install-dotnet-mac-rosetta:
+    steps:
+      - install-brew-rosetta
+      - run: |
+          curl -OL "https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh"
+          chmod a+x dotnet-install.sh
+          ./dotnet-install.sh --version 6.0.125
+          ln -s /Users/distiller/.dotnet/dotnet /usr/local/bin/
+
+  install-dotnet-linux:
+    steps:
+      - run: |
+          rpm -Uvh https://packages.microsoft.com/config/centos/7/packages-microsoft-prod.rpm
+          yum install -y dotnet-sdk-6.0
+
+  install-libicu-linux:
+    steps:
+      - run: yum install -y libicu60
+
+  deploy-dotnet-runtime-snapshot-unix:
+    steps:
+      - run: |
+          export DEPLOY_NUGET_API_KEY=$REPO_TYPEDB_PASSWORD
+          bazel run --jobs=8 --define version=0.0.0-$(git rev-parse HEAD) //csharp:driver-csharp-runtime-push -- snapshot
+
+  deploy-dotnet-snapshot-unix:
+    steps:
+      - run: |
+          export DEPLOY_NUGET_API_KEY=$REPO_TYPEDB_PASSWORD
+          bazel run --jobs=8 --define version=0.0.0-$(git rev-parse HEAD) //csharp:driver-csharp-push -- snapshot
+
+  test-dotnet-snapshot-unix:
+    steps:
+      - run: |
+          tool/test/start-core-server.sh
+          sleep 3
+          sed -i -e "s/DRIVER_CSHARP_VERSION_MARKER/0.0.0-$CIRCLE_SHA1/g" csharp/Test/Deployment/NugetApplicationTest.csproj
+          cat csharp/Test/Deployment/NugetApplicationTest.csproj
+          (cd csharp/Test/Deployment && dotnet run NugetApplicationTest.csproj)
+          tool/test/stop-core-server.sh
+
+  test-dotnet-snapshot-mac-rosetta:
+    steps:
+      - run: |
+          tool/test/start-core-server.sh
+          sleep 3
+          sed -i -e "s/DRIVER_CSHARP_VERSION_MARKER/0.0.0-$CIRCLE_SHA1/g" csharp/Test/Deployment/NugetApplicationTest.csproj
+          cat csharp/Test/Deployment/NugetApplicationTest.csproj
+          (cd csharp/Test/Deployment && /usr/local/bin/dotnet run NugetApplicationTest.csproj)
+          tool/test/stop-core-server.sh
+
+  deploy-dotnet-runtime-release-unix:
+    steps:
+      - run: |
+          export DEPLOY_NUGET_API_KEY=$REPO_NUGET_TOKEN
+          bazel run --jobs=8 --define version=$(cat VERSION) //csharp:driver-csharp-runtime-push --compilation_mode=opt -- release
+
+  deploy-dotnet-release-unix:
+    steps:
+      - run: |
+          export DEPLOY_NUGET_API_KEY=$REPO_NUGET_TOKEN
+          bazel run --jobs=8 --define version=$(cat VERSION) //csharp:driver-csharp-push --compilation_mode=opt -- release
+
   #########################
   # Rust deployment steps #
   #########################
@@ -430,6 +505,9 @@ jobs:
       - test-cpp-assembly-linux:
           target-arch: arm64
 
+      - install-libicu-linux
+      - deploy-dotnet-runtime-snapshot-unix
+
   deploy-snapshot-linux-x86_64:
     executor: linux-x86_64-amazonlinux-2
     steps:
@@ -449,6 +527,9 @@ jobs:
       - test-cpp-assembly-linux:
           target-arch: x86_64
 
+      - install-libicu-linux
+      - deploy-dotnet-runtime-snapshot-unix
+
   deploy-snapshot-mac-arm64:
     executor: mac-arm64
     steps:
@@ -470,6 +551,8 @@ jobs:
       - test-cpp-assembly-mac:
           target-arch: arm64
 
+      - deploy-dotnet-runtime-snapshot-unix
+
   deploy-snapshot-mac-x86_64:
     executor: mac-arm64
     steps:
@@ -492,6 +575,8 @@ jobs:
       - test-cpp-assembly-mac:
           target-arch: x86_64
 
+      - deploy-dotnet-runtime-snapshot-unix
+
   deploy-snapshot-windows-x86_64:
     executor:
       name: win/default
@@ -512,6 +597,8 @@ jobs:
       - run: .circleci\windows\cpp\deploy_snapshot.bat
       - run: .circleci\windows\cpp\test_assembly.bat
 
+      - run: .circleci\windows\csharp\deploy_snapshot.bat
+
   deploy-snapshot-any:
     executor: linux-x86_64-ubuntu-2204
     steps:
@@ -522,6 +609,7 @@ jobs:
       - deploy-maven-snapshot-unix
       - install-npm-apt
       - deploy-npm-snapshot-unix
+      - deploy-dotnet-snapshot-unix
 
   test-snapshot-linux-arm64:
     executor: linux-arm64-amazonlinux-2
@@ -532,6 +620,9 @@ jobs:
       - test-pip-snapshot-unix
       - install-maven-linux
       - test-maven-snapshot-unix
+      - install-dotnet-linux
+      - install-libicu-linux
+      - test-dotnet-snapshot-unix
 
   test-snapshot-linux-x86_64:
     executor: linux-x86_64-amazonlinux-2
@@ -542,6 +633,8 @@ jobs:
       - test-pip-snapshot-unix
       - install-maven-linux
       - test-maven-snapshot-unix
+      - install-dotnet-linux
+      - test-dotnet-snapshot-unix
 
   test-snapshot-mac-arm64:
     executor: mac-arm64
@@ -552,6 +645,8 @@ jobs:
       - test-pip-snapshot-unix
       - install-maven-mac
       - test-maven-snapshot-unix
+      - install-dotnet-mac
+      - test-dotnet-snapshot-unix
 
   test-snapshot-mac-x86_64:
     executor: mac-arm64
@@ -563,6 +658,8 @@ jobs:
       - test-pip-snapshot-mac-rosetta
       - install-maven-mac-rosetta
       - test-maven-snapshot-mac-rosetta
+      - install-dotnet-mac-rosetta
+      - test-dotnet-snapshot-mac-rosetta
 
   test-snapshot-windows-x86_64:
     executor:
@@ -575,6 +672,7 @@ jobs:
       - run: .circleci\windows\prepare.bat
       - run: .circleci\windows\python\test_deploy_snapshot.bat
       - run: .circleci\windows\java\test_deploy_snapshot.bat
+      - run: .circleci\windows\csharp\test_deploy_snapshot.bat
 
   test-snapshot-any:
     executor: linux-x86_64-ubuntu-2204
@@ -600,6 +698,9 @@ jobs:
       - deploy-clib-release-unix
       - deploy-cpp-release-unix
 
+      - install-libicu-linux
+      - deploy-dotnet-runtime-release-unix
+
   deploy-release-linux-x86_64:
     executor: linux-x86_64-amazonlinux-2
     steps:
@@ -611,6 +712,9 @@ jobs:
       - deploy-clib-release-unix
       - deploy-cpp-release-unix
 
+      - install-libicu-linux
+      - deploy-dotnet-runtime-release-unix
+
   deploy-release-mac-arm64:
     executor: mac-arm64
     steps:
@@ -621,6 +725,7 @@ jobs:
       - deploy-maven-jni-release-unix
       - deploy-clib-release-unix
       - deploy-cpp-release-unix
+      - deploy-dotnet-runtime-release-unix
 
   deploy-release-mac-x86_64:
     executor: mac-arm64
@@ -633,6 +738,7 @@ jobs:
       - deploy-maven-jni-release-unix
       - deploy-clib-release-unix
       - deploy-cpp-release-unix
+      - deploy-dotnet-runtime-release-unix
 
   deploy-release-windows-x86_64:
     executor:
@@ -647,6 +753,7 @@ jobs:
       - run: .circleci\windows\java\deploy_release.bat
       - run: .circleci\windows\clib\deploy_release.bat
       - run: .circleci\windows\cpp\deploy_release.bat
+      - run: .circleci\windows\csharp\deploy_release.bat
 
   deploy-release-any:
     executor: linux-x86_64-ubuntu-2204
@@ -658,6 +765,7 @@ jobs:
       - deploy-maven-release-unix
       - install-npm-apt
       - deploy-npm-release-unix
+      - deploy-dotnet-release-unix
 
   deploy-github:
     executor: linux-x86_64-ubuntu-2204
@@ -697,26 +805,29 @@ jobs:
           apt install -y git
           git push --delete https://[email protected]/vaticle/typedb-driver.git $CIRCLE_BRANCH
 
-
 workflows:
   typedb-driver-snapshot:
     jobs:
       - deploy-snapshot-linux-arm64:
           filters:
             branches:
               only: [development, master]
+
       - deploy-snapshot-linux-x86_64:
           filters:
             branches:
               only: [development, master]
+
       - deploy-snapshot-mac-arm64:
           filters:
             branches:
               only: [development, master]
+
       - deploy-snapshot-mac-x86_64:
           filters:
             branches:
               only: [development, master]
+
       - deploy-snapshot-windows-x86_64:
           filters:
             branches:
@@ -740,27 +851,31 @@ workflows:
           requires:
             - deploy-snapshot-linux-arm64
             - deploy-snapshot-any
+
       - test-snapshot-linux-x86_64:
           filters:
             branches:
               only: [master]
           requires:
             - deploy-snapshot-linux-x86_64
             - deploy-snapshot-any
+
       - test-snapshot-mac-arm64:
           filters:
             branches:
               only: [master]
           requires:
             - deploy-snapshot-mac-arm64
             - deploy-snapshot-any
+
       - test-snapshot-mac-x86_64:
           filters:
             branches:
               only: [master]
           requires:
             - deploy-snapshot-mac-x86_64
             - deploy-snapshot-any
+
       - test-snapshot-windows-x86_64:
           filters:
             branches:

diff --git a/.circleci/windows/clib/deploy_release.bat b/.circleci/windows/clib/deploy_release.bat
@@ -25,5 +25,5 @@ SET DEPLOY_ARTIFACT_USERNAME=%REPO_TYPEDB_USERNAME%
 SET DEPLOY_ARTIFACT_PASSWORD=%REPO_TYPEDB_PASSWORD%
 
 SET /p VER=<VERSION
-bazel --output_user_root=C:\bazel run --verbose_failures --define version=%VER% //c:deploy-windows-x86_64-zip --compilation_mode=opt -- release
+bazel --output_user_root=C:\b run --verbose_failures --define version=%VER% //c:deploy-windows-x86_64-zip --compilation_mode=opt -- release
 IF %errorlevel% NEQ 0 EXIT /b %errorlevel%
diff --git a/.circleci/windows/clib/deploy_snapshot.bat b/.circleci/windows/clib/deploy_snapshot.bat
@@ -25,5 +25,5 @@ SET DEPLOY_ARTIFACT_PASSWORD=%REPO_TYPEDB_PASSWORD%
 
 git rev-parse HEAD > version_snapshot.txt
 set /p VER=<version_snapshot.txt
-bazel --output_user_root=C:\bazel run --verbose_failures --define version=%VER% //c:deploy-windows-x86_64-zip -- snapshot
+bazel --output_user_root=C:\b run --verbose_failures --define version=%VER% //c:deploy-windows-x86_64-zip -- snapshot
 IF %errorlevel% NEQ 0 EXIT /b %errorlevel%
diff --git a/.circleci/windows/clib/test_assembly.bat b/.circleci/windows/clib/test_assembly.bat
@@ -21,12 +21,12 @@ choco install 7zip.portable --limit-output --yes --no-progress
 choco install cmake.install --version 3.27.0 --installargs '"ADD_CMAKE_TO_PATH=User"' --limit-output --yes --no-progress
 CALL refreshenv
 
-bazel --output_user_root=C:\bazel build @vaticle_typedb_artifact_windows-x86_64//file
+bazel --output_user_root=C:\b build @vaticle_typedb_artifact_windows-x86_64//file
 powershell -Command "Move-Item -Path bazel-typedb-driver\external\vaticle_typedb_artifact_windows-x86_64\file\typedb-server-windows* -Destination typedb-server-windows.zip"
 7z x typedb-server-windows.zip
 powershell -Command "Move-Item -Path typedb-server-windows-* -Destination typedb-server-windows"
 
-bazel --output_user_root=C:\bazel build //c:assemble-windows-x86_64-zip
+bazel --output_user_root=C:\b build //c:assemble-windows-x86_64-zip
 mkdir test_assembly_clib
 pushd test_assembly_clib
 7z x ..\bazel-bin\c\typedb-driver-clib-windows-x86_64.zip

diff --git a/.circleci/windows/cpp/deploy_release.bat b/.circleci/windows/cpp/deploy_release.bat
@@ -25,5 +25,5 @@ SET DEPLOY_ARTIFACT_USERNAME=%REPO_TYPEDB_USERNAME%
 SET DEPLOY_ARTIFACT_PASSWORD=%REPO_TYPEDB_PASSWORD%
 
 SET /p VER=<VERSION
-bazel --output_user_root=C:\bazel run --verbose_failures --define version=%VER% //cpp:deploy-windows-x86_64-zip --compilation_mode=opt -- release
+bazel --output_user_root=C:\b run --verbose_failures --define version=%VER% //cpp:deploy-windows-x86_64-zip --compilation_mode=opt -- release
 IF %errorlevel% NEQ 0 EXIT /b %errorlevel%
diff --git a/.circleci/windows/cpp/deploy_snapshot.bat b/.circleci/windows/cpp/deploy_snapshot.bat
@@ -25,5 +25,5 @@ SET DEPLOY_ARTIFACT_PASSWORD=%REPO_TYPEDB_PASSWORD%
 
 git rev-parse HEAD > version_snapshot.txt
 set /p VER=<version_snapshot.txt
-bazel --output_user_root=C:\bazel run --verbose_failures --define version=%VER% //cpp:deploy-windows-x86_64-zip -- snapshot
+bazel --output_user_root=C:\b run --verbose_failures --define version=%VER% //cpp:deploy-windows-x86_64-zip -- snapshot
 IF %errorlevel% NEQ 0 EXIT /b %errorlevel%
diff --git a/.circleci/windows/cpp/test_assembly.bat b/.circleci/windows/cpp/test_assembly.bat
@@ -21,12 +21,12 @@ choco install 7zip.portable --limit-output --yes --no-progress
 choco install cmake.install --version 3.27.0 --installargs '"ADD_CMAKE_TO_PATH=User"' --limit-output --yes --no-progress
 CALL refreshenv
 
-bazel --output_user_root=C:\bazel build @vaticle_typedb_artifact_windows-x86_64//file
+bazel --output_user_root=C:\b build @vaticle_typedb_artifact_windows-x86_64//file
 powershell -Command "Move-Item -Path bazel-typedb-driver\external\vaticle_typedb_artifact_windows-x86_64\file\typedb-server-windows* -Destination typedb-server-windows.zip"
 7z x typedb-server-windows.zip
 powershell -Command "Move-Item -Path typedb-server-windows-* -Destination typedb-server-windows"
 
-bazel --output_user_root=C:\bazel build //cpp:assemble-windows-x86_64-zip
+bazel --output_user_root=C:\b build //cpp:assemble-windows-x86_64-zip
 mkdir test_assembly_cpp
 pushd test_assembly_cpp
 7z x ..\bazel-bin\cpp\typedb-driver-cpp-windows-x86_64.zip

diff --git a/.circleci/windows/csharp/deploy_release.bat b/.circleci/windows/csharp/deploy_release.bat
@@ -0,0 +1,28 @@
+@echo off
+REM Licensed to the Apache Software Foundation (ASF) under one
+REM or more contributor license agreements.  See the NOTICE file
+REM distributed with this work for additional information
+REM regarding copyright ownership.  The ASF licenses this file
+REM to you under the Apache License, Version 2.0 (the
+REM "License"); you may not use this file except in compliance
+REM with the License.  You may obtain a copy of the License at
+REM
+REM   http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing,
+REM software distributed under the License is distributed on an
+REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+REM KIND, either express or implied.  See the License for the
+REM specific language governing permissions and limitations
+REM under the License.
+
+REM needs to be called such that software installed
+REM by Chocolatey in prepare.bat is accessible
+CALL refreshenv
+
+ECHO Building and deploying windows package...
+SET DEPLOY_NUGET_API_KEY=%REPO_NUGET_TOKEN%
+
+SET /p VER=<VERSION
+bazel --output_user_root=C:/b run --verbose_failures --define version=%VER% //csharp:driver-csharp-runtime-push --compilation_mode=opt -- release
+IF %errorlevel% NEQ 0 EXIT /b %errorlevel%