Skip to content

Commit

Permalink
Feature/user dev guides (#31)
Browse files Browse the repository at this point in the history
* feature: userguide created

* Updated user guide with improved explanations

* Added explanation on how to add new parameters to the participant configuration

* remove table of contents and developerguide extended

* Added datasets and aggregator guide

* fix port definition

* expanded user and developer guides with more details

* added dropdown for easier access to guides

* simplified installation guide

* Improve documentation formatting and readability

* Improve installation guide

* fix subindices

---------

Co-authored-by: enriquetomasmb <[email protected]>
Co-authored-by: Enrique Tomás <[email protected]>
  • Loading branch information
3 people authored Feb 4, 2025
1 parent 3e8353e commit 0c14f56
Show file tree
Hide file tree
Showing 21 changed files with 1,239 additions and 80 deletions.
4 changes: 4 additions & 0 deletions docs/_prebuilt/changelog.md
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
---
hide:
-toc
---
--8<-- "CHANGELOG.md"
4 changes: 4 additions & 0 deletions docs/_prebuilt/contributing.md
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
---
hide:
-toc
---
--8<-- "CONTRIBUTING.md"
7 changes: 7 additions & 0 deletions docs/_prebuilt/css/toc.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.md-sidebar--secondary {
order: 0;
}

.md-sidebar--primary {
display: none;
}
819 changes: 819 additions & 0 deletions docs/_prebuilt/developerguide.md

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions docs/_prebuilt/index.md
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
---
hide:
-toc
---
--8<-- "README.md"
134 changes: 56 additions & 78 deletions docs/_prebuilt/installation.md
Original file line number Diff line number Diff line change
@@ -1,136 +1,114 @@
# Installation
# **Installation**

In this section, we will explain how to install the NEBULA platform.
Welcome to the NEBULA platform installation guide. This document explains how to obtain, install, run, and troubleshoot NEBULA.

## Table of contents
## **Prerequisites**

- [Prerequisites](#prerequisites)
- [Obtaining NEBULA platform](#obtaining-nebula-platform)
- [Installing NEBULA platform](#installing-nebula-platform)
- [Checking the installation](#checking-the-installation)
- [Building NEBULA core](#building-nebula-core)
- [Running NEBULA](#running-nebula)
- [NEBULA Frontend](#nebula-frontend)
- [Stop NEBULA](#stop-nebula)
- [Possible issues during the installation or execution](#possible-issues-during-the-installation-or-execution)
For the best experience, ensure the following prerequisites are met:

## Prerequisites

- Python 3.10 or higher (3.11 recommended)
- Docker Engine 24.0.4 or higher (24.0.7 recommended, https://docs.docker.com/engine/install/)
- Docker Compose 2.19.0 or higher (2.19.1 recommended, https://docs.docker.com/compose/install/)
- For using NVIDIA GPUs, NVIDIA driver version >=525.60.13 (Linux) >=528.33 (Windows), and CUDA 12.1 (mandatory). More information in https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html (Windows) or https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html (Linux)

## Obtaining NEBULA platform
- **Linux** (Ubuntu 20.04 LTS recommended) or **macOS** (10.15 Catalina or later). Currently, we do not maintain an up-to-date version for Windows.
- Minimum **8 GB RAM** (+32 GB recommended for virtualized devices).
- Minimum **20 GB disk space** for Docker images and containers. Additional space is required for datasets, models, and results.
- **Docker Engine** 24.0.4 or higher (24.0.7 recommended, https://docs.docker.com/engine/install/)
- **Docker Compose** 2.19.0 or higher (2.19.1 recommended, https://docs.docker.com/compose/install/)

## **Obtaining NEBULA**

You can obtain the source code from https://github.com/CyberDataLab/nebula

Or, if you happen to have git configured, you can clone the repository:
Or clone the repository using git:

git clone https://github.com/CyberDataLab/nebula.git
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">git clone https://github.com/CyberDataLab/nebula.git</span></code></pre>

Now, you can move to the source directory:

cd nebula

### Installing NEBULA platform

To install the NEBULA platform, you can use the following command line:

make install
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">cd nebula</span></code></pre>

This command will install the required dependencies for the platform and open a shell to start the platform.
### **Installing NEBULA**

If the shell is not opened or you want to reopen the shell, you can use the following command line:
Install required dependencies and set up Docker containers by running:

make shell
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">make install</span></code></pre>

### Checking the installation
Next, activate the virtual environment:

Once the installation is finished, you can check by listing the version
of the NEBULA with the following command line:
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">source .venv/bin/activate</span></code></pre>

python app/main.py --version
If you forget this command, you can type:

## Building NEBULA core
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">make shell</span></code></pre>

There are two ways to deploy the node in the federation: using Docker
containers or isolated processes. You can choose the one that best fits
your needs in the frontend.
Your shell prompt should look similar to:

### 1. Using Docker containers
<pre><code><span style="color: grey;">(nebula-dfl)</span> <span style="color: blue;">user@host</span>:~$</code></pre>

You need to build the docker image using the following command line in
the root directory:
### **Using NVIDIA GPU on Nodes (Optional)**

docker build -t nebula-core .
For nodes equipped with **NVIDIA GPUs**, ensure the following prerequisites:

In case of using GPU in the docker, you have to follow the instructions
in the following link to install nvidia-container-toolkit:
- **NVIDIA Driver**: Version 525.60.13 or later.
- **CUDA**: Version 12.1 is required. After installation, verify with <code>nvidia-smi</code>.
- **NVIDIA Container Toolkit**: Install to enable GPU access within Docker containers.

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html
Follow these guides for proper installation:

You can check the docker images using the following command line:
- [CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)
- [NVIDIA Container Toolkit Installation Guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)

docker images
Note: Ensure that the CUDA toolkit version is compatible with your driver and, if needed, update the Docker runtime to support GPU integration.

### 2. Using isolated processes
## **Running NEBULA**

There is no need additional steps to build the core using isolated processes.
Once the installation is finished, you can check if NEBULA is installed properly using:

## Running NEBULA
<pre><code><span style="color: grey;">(nebula-dfl) </span><span style="color: blue;">user@host</span>:~$ <span style="color: green;">python app/main.py --version</span></code></pre>

To run NEBULA, you can use the following command line:

python app/main.py [PARAMS]
<pre><code><span style="color: grey;">(nebula-dfl)</span> <span style="color: blue;">user@host</span>:~$ <span style="color: green;">python app/main.py</span></code></pre>

The first time you run the platform, the nebula-frontend docker image
will be built. This process can take a few minutes.
Note: The first run may build the nebula-frontend Docker image, which can take a few minutes.

You can show the PARAMS using:
Display available parameters:

python app/main.py --help
<pre><code><span style="color: grey;">(nebula-dfl)</span> <span style="color: blue;">user@host</span>:~$ <span style="color: green;">python app/main.py --help</span></code></pre>

The frontend will be available at http://127.0.0.1:6060 (by default)
By default, the frontend is available at http://127.0.0.1:6060. If the 6060 port is unavailable, a random port will be assigned automatically and prompted in the console.

To change the default port of the frontend, you can use the following
command line:
Also, you can define the specific port using the following command line:

python app/main.py --webport [PORT]
<pre><code><span style="color: grey;">(nebula-dfl)</span> <span style="color: blue;">user@host</span>:~$ <span style="color: green;">python app/main.py --webport [PORT]</span></code></pre>

To change the default port of the statistics endpoint, you can use the
following command line:
and the default port of the statistics endpoint:

python app/main.py --statsport [PORT]
<pre><code><span style="color: grey;">(nebula-dfl)</span> <span style="color: blue;">user@host</span>:~$ <span style="color: green;">python app/main.py --statsport [PORT]</span></code></pre>

## NEBULA Frontend
## **NEBULA Frontend Credentials**

You can login with the following credentials:
You can log in with the default credentials:

- User: admin
- Password: admin

If not working the default credentials, send an email to [Enrique Tomás
Martínez Beltrán](mailto:[email protected]) to get the credentials.
If these do not work, please contact Enrique Tomás Martínez Beltrán at [[email protected]](mailto:[email protected]).

## Stop NEBULA
## **Stopping NEBULA**

To stop NEBULA, you can use the following command line:

python app/main.py --stop
<pre><code><span style="color: grey;">(nebula-dfl)</span> <span style="color: blue;">user@host</span>:~$ <span style="color: green;">python app/main.py --stop</span></code></pre>

Be careful, this command will stop all the containers related to NEBULA:
frontend, controller, and nodes.
Be careful! This command will stop all the containers related to NEBULA:
Frontend, Controller, and Nodes.

## Possible issues during the installation or execution
## **Troubleshooting**

If frontend is not working, check the logs in app/logs/frontend.log

If any of the following errors appear, take a look at the docker logs of
the nebula-frontend container:

docker logs nebula-frontend
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">docker logs user_nebula-frontend</span></code></pre>

------------------------------------------------------------------------

Expand All @@ -139,7 +117,7 @@ from daemon: Pool overlaps with other one on this address space

Solution: Delete the docker network nebula_X

> docker network rm nebula_X
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">docker network rm nebula_X</span></code></pre>

------------------------------------------------------------------------

Expand All @@ -148,7 +126,7 @@ unix:///var/run/docker.sock. Is the docker daemon running?

Solution: Start the docker daemon

> sudo dockerd
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">sudo dockerdX</span></code></pre>

Solution: Enable the following option in Docker Desktop

Expand All @@ -163,13 +141,13 @@ docker daemon running?

Solution: Start the docker daemon

> sudo dockerd -H tcp://X.X.X.X:2375
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">sudo dockerd -H tcp://X.X.X.X:2375</span></code></pre>

------------------------------------------------------------------------

If frontend is not working, restart docker daemon

> sudo systemctl restart docker
<pre><code><span style="color: blue;">user@host</span>:~$ <span style="color: green;">sudo systemctl restart docker</span></code></pre>

------------------------------------------------------------------------

Expand Down
5 changes: 5 additions & 0 deletions docs/_prebuilt/js/toc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
document.addEventListener('DOMContentLoaded', function() {
if (window.location.pathname.includes("api")) {
document.querySelector('.md-sidebar--primary').style.display = 'block';
}
});
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_prebuilt/static/userguide/deployment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_prebuilt/static/userguide/frontend.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_prebuilt/static/userguide/monitor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file removed docs/_prebuilt/usage.md
Empty file.
Loading

0 comments on commit 0c14f56

Please sign in to comment.