change: Converted the SQLAlchemy-related code to use async/await. (#618)

* Updated Flask and SQLAlchemy requirements to the async version.

* Added the AsyncAttrs mixin to the models.

* Added the async driver for Postgres.

* Set the posts' user relationship to 'selectin' for the join.

* Switched the roles' and permissions' relationship joins to 'selectin'.

* Added async counterparts to all methods in the DB class.

* Added missing deprecation warning.

* Updated all endpoints to use the new async methods.

* Installed Quart and Quart-cors to replace Flask (and Flask-cors).

* Switched Auth to use async/await.

* Replaced Flask and Flask-cors with Quart and Quart-cors.

* Deleted most of the non-async methods.

* Installed pytest-asyncio for tests.

* Updated the tests to use the async versions of all functions.

* Deleted the rest of the non-async functions in db.

* Cleaned up some of the test fixtures.

* Merged the two app fixtures.

Now that we're not using the app context anymore, we don't really need separate fixtures for the app and the test_client.

* Split the db setup fixtures.

* Added missing role to test setup,

* Replaced daatabase_url with async_database_url in the DB class setup.

* Tweaked the fixtures a little more.

* Changed asyncio_mode to auto in tests.

* Changed the asyncio scope to session.

* Updated the URL of the database in CI to include the driver.

* Added a subscriptions task in tests.

* Updated the way the database is reset in tests.

* Updated the docstrings.

* Deleted sh from the README as we no longer use it.

* Disabled sending push notifications in tests.

* Moved the mock to the function-scoped fixture.

* Added some logging to see what the problem is.

* Testing to see if it's the wrong location to patch.

* Deleted the unneeded print.

* Added a changelog entry.

* Changed all db methods to return formatted objects.

* Added missing parsing of dates to all dates coming from the front-end.

* Cast all the IDs explicitly to integers.

Seems that asyncpg isn't as good at handling str -> int as psycopg2...

* Fixed the bug I caused in the GET users endpoint.

* Added refreshes to the 'add' methods in the db.

* Updated the db tests to the formatted return value.

* Updated the date format in the dummy data to match the front-end.

* Cast some more string IDs as integers...

* Updated the error message in test_db to match the asyncpg syntax.

* Updated the fixture in test_auth to the db fixture.

* Fixed some issues with the fixtures setting up the database.

* Added proepr logging to the db class.

* Changed the name of the app.

* Added a note to the README to include the driver in the URL.

* Added missing engine.dispose to the suite setup.

* Deleted the old changelog file.

* Replaced gunicorn with hypercorn and removed psycopg as it's not needed anymore.

* Changed the names of the db's attributes and methods back to their original names.

They were originally named this way to avoid breaking any other methods that use the originals during the migration, but now that everything has been migrated and the old methods have been removed, the changed names are no longer needed.

* Added a changelog entry.

* Updated the date format in the API docs.

* Fixed a 403 for user update.

* Fixed the tests I broke...

.circleci/config.yml

@@ -12,7 +12,7 @@ jobs:
       - image: cimg/python:3.11
         environment:
           FLASK_CONFIG: testing
-          TEST_DATABASE_URL: postgresql://postgres:password@localhost:5432/test_sah?sslmode=disable
+          TEST_DATABASE_URL: postgresql+asyncpg://postgres:password@localhost:5432/test_sah?sslmode=disable
     steps:
       - checkout
       - restore_cache:
@@ -30,7 +30,7 @@ jobs:
       - image: cimg/python:3.11
         environment:
           FLASK_CONFIG: testing
-          TEST_DATABASE_URL: postgresql://postgres:password@localhost:5432/test_sah?sslmode=disable
+          TEST_DATABASE_URL: postgresql+asyncpg://postgres:password@localhost:5432/test_sah?sslmode=disable
       - image: cimg/postgres:14.10
         auth:
           username: mydockerhub-user

API_DOCS.md

@@ -251,7 +251,7 @@ For full instructions check the [`backend README`](./backend/README.md)
 **Expected Errors**:
   - 500 (Internal Server Error) - In case there's an error adding the new post to the database.
 
-**CURL Request Sample**: `curl -X POST http://127.0.0.1:5000/posts -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"user_id":4, "user":"user14", "text":"test curl", "date":"Wed Jun 10 2020 10:30:05 GMT+0300", "givenHugs":0}'`
+**CURL Request Sample**: `curl -X POST http://127.0.0.1:5000/posts -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"user_id":4, "user":"user14", "text":"test curl", "date":"2020-06-07T15:57:45.901Z", "givenHugs":0}'`
 
 **Response Example:**
 ```
@@ -290,7 +290,7 @@ For full instructions check the [`backend README`](./backend/README.md)
   - 404 (Not Found) - In case there's no post with that ID.
   - 500 (Internal Server Error) - In case there's an error updating the post's data in the database.
 
-**CURL Request Sample**: `curl -X PATCH http://127.0.0.1:5000/posts/15 -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"user_id":4, "user":"user14", "text":"test curl", "date":"Wed Jun 10 2020 10:30:05 GMT+0300", "givenHugs":0}'`
+**CURL Request Sample**: `curl -X PATCH http://127.0.0.1:5000/posts/15 -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"user_id":4, "user":"user14", "text":"test curl", "date":"2020-06-07T15:57:45.901Z", "givenHugs":0}'`
 
 **Response Example:**
 ```
@@ -750,7 +750,7 @@ For full instructions check the [`backend README`](./backend/README.md)
   - 403 (Forbidden) - In case the user is trying to post the message from another user.
   - 500 (Internal Server Error) - In case there's an error adding the new message (or the new thread if one is needed) to the database.
 
-**CURL Request Sample**: `curl -X POST http://127.0.0.1:5000/messages -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"from":"user14", "fromId":4, "forId":1, "messageText":"hang in there", "date":"Mon, 08 Jun 2020 14:43:15 GMT"}'`
+**CURL Request Sample**: `curl -X POST http://127.0.0.1:5000/messages -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"from":"user14", "fromId":4, "forId":1, "messageText":"hang in there", "date":"2020-06-07T15:57:45.901Z"}'`
 
 **Response Example:**
 ```
@@ -894,7 +894,7 @@ For full instructions check the [`backend README`](./backend/README.md)
   - 404 (Not Found) - In case the item being reported doesn't exist.
   - 500 (Internal Server Error) - In case an error occurred while adding the new report to the database.
 
-**CURL Request Sample**: `curl -X POST http://127.0.0.1:5000/reports -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"type":"Post", "userID":"1", "postID":4, "reporter":5,"reportReason":"this post is inappropriate", "date":"Tue Jun 23 2020 14:59:31 GMT+0300"}'`
+**CURL Request Sample**: `curl -X POST http://127.0.0.1:5000/reports -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"type":"Post", "userID":"1", "postID":4, "reporter":5,"reportReason":"this post is inappropriate", "date":"2020-06-07T15:57:45.901Z"}'`
 
 **Response Example:**
 ```
@@ -942,7 +942,7 @@ For full instructions check the [`backend README`](./backend/README.md)
   - 404 (Not Found) - In case a report with that ID doesn't exist.
   - 500 (Internal Server Error) - In case an error occurred while updating the report in the database.
 
-**CURL Request Sample**: `curl -X PATCH http://127.0.0.1:5000/reports/36 -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"type":"Post", "userID":"1", "postID":4, "reporter":5,"reportReason":"this post is inappropriate", "date":"Tue Jun 23 2020 14:59:31 GMT+0300", "dismissed": true, "closed": true}'`
+**CURL Request Sample**: `curl -X PATCH http://127.0.0.1:5000/reports/36 -H "Content-Type: application/json" -H 'Authorization: Bearer <YOUR_TOKEN>' -d '{"type":"Post", "userID":"1", "postID":4, "reporter":5,"reportReason":"this post is inappropriate", "date":"2020-06-07T15:57:45.901Z", "dismissed": true, "closed": true}'`
 
 **Response Example:**
 ```

README.md

@@ -35,7 +35,7 @@ The project is open source, so feel free to use parts of the code. However, the
 5. Run ```pre-commit install``` to install and initialise pre-commit.
 6. Create a database for the app.
 7. Update the database URI to match your system.
-    - The database URI comes from an environment variable named **DATABASE_URL**.
+    - The database URI comes from an environment variable named **DATABASE_URL**. Make sure you include the driver in the URL (e.g., `postgresql+asyncpg` instead of `postgresql`), as otherwise SQLAlchemy assumes it should use the default driver, which (at least for postgres) doesn't support async/await.
 8. Set Auth0 configuration variables:
     - AUTH0_DOMAIN - environment variable containing your Auth0 domain.
     - API_AUDIENCE - environment variable containing your Auth0 API audience.

auth.py

@@ -250,13 +250,15 @@ def check_permissions_legacy(permission: list[str], payload: dict[str, Any]) ->
     return True
 
 
-def get_current_user(payload: dict[str, Any], db: SendADatabase) -> dict[str, Any]:
+async def get_current_user(
+    payload: dict[str, Any], db: SendADatabase
+) -> dict[str, Any]:
     """
     Fetches the details of the currently logged in user from the database.
 
     param payload: The payload from the decoded, verified JWT.
     """
-    current_user: User | None = db.session.scalar(
+    current_user: User | None = await db.session.scalar(
         select(User).filter(User.auth0_id == payload["sub"])
     )
 
@@ -323,7 +325,7 @@ async def wrapper(*args, **kwargs):
                 returned_payload = payload
                 check_permissions_legacy(permission, payload)
             else:
-                current_user = get_current_user(payload, db)
+                current_user = await get_current_user(payload, db)
                 returned_payload = {
                     "id": current_user["id"],
                     "auth0Id": current_user["auth0Id"],

changelog/pr618.json

@@ -0,0 +1,37 @@
+{
+  "pr_number": 618,
+  "changes": [
+    {
+      "change": "Changes",
+      "description": "Updated all database interactions to use SQLAlchemy's asyncio extension. The engine, session maker and session have been replaced by their async variants, and the helper methods for interacting with the database have all been updated to use async/await. This means that we can now write asynchronous code for read, write, update and delete, which should allow the server itself to handle more requests at once."
+    },
+    {
+      "change": "Changes",
+      "description": "The helper methods for creating and updating objects now return the formatted object instead of the raw SQLAlchemy object."
+    },
+    {
+      "change": "Features",
+      "description": "Added a logger to the Database Handler class and added logging for all errors raised by the database."
+    },
+    {
+      "change": "Chores",
+      "description": "Replaced psycopg2 with asyncpg as the driver used for database interactions, as psycopg2 doesn't support async/await."
+    },
+    {
+      "change": "Changes",
+      "description": "Changed several relationships (user relationship in posts; role relationship in users; and permission relationship in roles) to use eager loading using 'selectin' instead of the default lazy loading. The SQLAlchemy asyncio extension doesn't support accessing attributes of lazily-loaded objects, and all three attributes are required to return the formatted versions of the models they're defined in (post, user and role respectively)."
+    },
+    {
+      "change": "Fixes",
+      "description": "Dates are now explicitly parsed from strings passed into the API (using `strptime`) and strings that contain integers (e.g., IDs) are explicitly cast from strings to integers. This fixes an issue where inserts and queries errored due to the wrong type being used for dates/IDs."
+    },
+    {
+      "change": "Changes",
+      "description": "Changed the name of the Quart app from the current file's name to `SendAHug` to better reflect the server's purpose."
+    },
+    {
+      "change": "Fixes",
+      "description": "Fixed a bug where an attempt to update user details (such as login count) returned a 403 error saying 'you don't have permission to block users'. This happened due to a recent change in the front-end, which now sends the whole user object (of the logged in user) back to the back-end on update (instead of just the updated fields). As the back-end was checking whether the 'blocked' field existed, instead of whether it changed, this caused that error to be raised. Now, the back-end checks whether the value of 'blocked' changed instead."
+    }
+  ]
+}