Merge pull request 'Correction of the doc' (#2) from push-zkpmmkmzmzwl into master
Reviewed-on: #2
This commit was merged in pull request #2.
This commit is contained in:
183
Readme.md
183
Readme.md
@@ -1,13 +1,27 @@
|
|||||||
# JupyterHub Configuration with OrbStack on Mac (all in Docker)
|
# JupyterHub Configuration with OrbStack on Mac (all in Docker)
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
- OrbStack installed and running
|
|
||||||
|
|
||||||
## File Structure
|
You must have docker running on your computer
|
||||||
|
|
||||||
Your `~/jupyterhub-tp` directory should contain:
|
- On MacOS, [OrbStack](https://orbstack.dev/ "A Docker implementation optimised for MacOS") is recommanded
|
||||||
|
|
||||||
|
##
|
||||||
|
|
||||||
|
## Installation Steps
|
||||||
|
|
||||||
|
### 1. Create Directory Structure
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
git clone https://forge.metabarcoding.org/MetabarcodingSchool/OBIJupyterHub.git
|
||||||
```
|
```
|
||||||
~/jupyterhub-tp/
|
|
||||||
|
#### File Structure
|
||||||
|
|
||||||
|
Your `~/OBIJupyterHub` directory should contain:
|
||||||
|
|
||||||
|
```
|
||||||
|
~/OBIJupyterHub/
|
||||||
├── Dockerfile # Image for students (already created)
|
├── Dockerfile # Image for students (already created)
|
||||||
├── Dockerfile.hub # Image for JupyterHub (new)
|
├── Dockerfile.hub # Image for JupyterHub (new)
|
||||||
├── jupyterhub_config.py # Configuration
|
├── jupyterhub_config.py # Configuration
|
||||||
@@ -15,37 +29,13 @@ Your `~/jupyterhub-tp` directory should contain:
|
|||||||
└── start-jupyterhub.sh # Startup script
|
└── start-jupyterhub.sh # Startup script
|
||||||
```
|
```
|
||||||
|
|
||||||
## Installation Steps
|
### 2. Start JupyterHub
|
||||||
|
|
||||||
### 1. Create Directory Structure
|
``` bash
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p ~/jupyterhub-tp
|
|
||||||
cd ~/jupyterhub-tp
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Create All Necessary Files
|
|
||||||
|
|
||||||
Create the following files with the content from artifacts:
|
|
||||||
- `Dockerfile` (artifact "Dockerfile for JupyterHub with R and Bash")
|
|
||||||
- `Dockerfile.hub` (artifact "Dockerfile for JupyterHub container")
|
|
||||||
- `jupyterhub_config.py` (artifact "JupyterHub Configuration")
|
|
||||||
- `docker-compose.yml` (artifact "docker-compose.yml")
|
|
||||||
- `start-jupyterhub.sh` (artifact "start-jupyterhub.sh")
|
|
||||||
|
|
||||||
### 3. Make Startup Script Executable
|
|
||||||
|
|
||||||
```bash
|
|
||||||
chmod +x start-jupyterhub.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Start JupyterHub
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./start-jupyterhub.sh
|
./start-jupyterhub.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. Access JupyterHub
|
### 3. Access JupyterHub
|
||||||
|
|
||||||
Open your browser and go to: **http://localhost:8888**
|
Open your browser and go to: **http://localhost:8888**
|
||||||
|
|
||||||
@@ -54,27 +44,32 @@ You can log in with any username and password: `metabar2025`
|
|||||||
## Useful Commands
|
## Useful Commands
|
||||||
|
|
||||||
### View JupyterHub logs
|
### View JupyterHub logs
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
docker-compose logs -f jupyterhub
|
docker-compose logs -f jupyterhub
|
||||||
```
|
```
|
||||||
|
|
||||||
### View all containers (hub + students)
|
### View all containers (hub + students)
|
||||||
```bash
|
|
||||||
docker ps
|
``` bash
|
||||||
|
docker ps | grep jupyterhub
|
||||||
```
|
```
|
||||||
|
|
||||||
### Stop JupyterHub
|
### Stop JupyterHub
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
docker-compose down
|
docker-compose down
|
||||||
```
|
```
|
||||||
|
|
||||||
### Restart JupyterHub (after config modification)
|
### Restart JupyterHub (after config modification)
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
docker-compose restart jupyterhub
|
docker-compose restart jupyterhub
|
||||||
```
|
```
|
||||||
|
|
||||||
### Rebuild after Dockerfile modification
|
### Rebuild after Dockerfile modification
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
# For student image
|
# For student image
|
||||||
docker build -t jupyterhub-student:latest -f Dockerfile .
|
docker build -t jupyterhub-student:latest -f Dockerfile .
|
||||||
docker-compose restart jupyterhub
|
docker-compose restart jupyterhub
|
||||||
@@ -84,12 +79,16 @@ docker-compose up -d --build
|
|||||||
```
|
```
|
||||||
|
|
||||||
### View logs for a specific student
|
### View logs for a specific student
|
||||||
```bash
|
|
||||||
docker logs jupyter-username
|
``` bash
|
||||||
|
docker logs jupyter-<username>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Replace `<username>` by the actual user name of the student.
|
||||||
|
|
||||||
### Clean up after lab
|
### Clean up after lab
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
# Stop and remove all containers
|
# Stop and remove all containers
|
||||||
docker-compose down
|
docker-compose down
|
||||||
|
|
||||||
@@ -110,7 +109,8 @@ docker volume prune -f
|
|||||||
### Directory Structure for Each Student
|
### Directory Structure for Each Student
|
||||||
|
|
||||||
Each student will see this directory structure in their JupyterLab (everything under `work/` is persistent):
|
Each student will see this directory structure in their JupyterLab (everything under `work/` is persistent):
|
||||||
```
|
|
||||||
|
```
|
||||||
work/ # Personal workspace root (persistent)
|
work/ # Personal workspace root (persistent)
|
||||||
├── [student files] # Their own files and notebooks
|
├── [student files] # Their own files and notebooks
|
||||||
├── R_packages/ # Personal R packages (writable by student)
|
├── R_packages/ # Personal R packages (writable by student)
|
||||||
@@ -121,46 +121,34 @@ work/ # Personal workspace root (persistent)
|
|||||||
└── [course materials] # Your course files
|
└── [course materials] # Your course files
|
||||||
```
|
```
|
||||||
|
|
||||||
**R Package Priority:**
|
**R Package Priority:** 1. R checks `work/R_packages/` first (personal, writable) 2. Then `work/course/R_packages/` (shared, read-only, installed by prof) 3. Then system libraries
|
||||||
1. R checks `work/R_packages/` first (personal, writable)
|
|
||||||
2. Then `work/course/R_packages/` (shared, read-only, installed by prof)
|
|
||||||
3. Then system libraries
|
|
||||||
|
|
||||||
**Important:** Everything is under `work/`, so all student files are automatically saved in their persistent volume.
|
**Important:** Everything is under `work/`, so all student files are automatically saved in their persistent volume.
|
||||||
|
|
||||||
### User Accounts
|
### User Accounts
|
||||||
|
|
||||||
**Admin Account:**
|
**Admin Account:** - Username: `admin` - Password: `admin2025` (change in docker-compose.yml: `JUPYTERHUB_ADMIN_PASSWORD`) - Can write to `course/` directory
|
||||||
- Username: `admin`
|
|
||||||
- Password: `admin2025` (change in docker-compose.yml: `JUPYTERHUB_ADMIN_PASSWORD`)
|
|
||||||
- Can write to `course/` directory
|
|
||||||
|
|
||||||
**Student Accounts:**
|
**Student Accounts:** - Username: any name - Password: `metabar2025` (change in docker-compose.yml: `JUPYTERHUB_PASSWORD`) - Read-only access to `course/` directory
|
||||||
- Username: any name
|
|
||||||
- Password: `metabar2025` (change in docker-compose.yml: `JUPYTERHUB_PASSWORD`)
|
|
||||||
- Read-only access to `course/` directory
|
|
||||||
|
|
||||||
### Installing R Packages (Admin Only)
|
### Installing R Packages (Admin Only)
|
||||||
|
|
||||||
**From your Mac (recommended):**
|
**From your Mac (recommended):**
|
||||||
|
|
||||||
```bash
|
``` bash
|
||||||
chmod +x install-r-packages-admin.sh
|
chmod +x install-r-packages-admin.sh
|
||||||
|
|
||||||
# Install packages
|
# Install packages
|
||||||
./install-r-packages-admin.sh reshape2 plotly knitr
|
./install-r-packages-admin.sh reshape2 plotly knitr
|
||||||
```
|
```
|
||||||
|
|
||||||
This script:
|
This script: - Installs packages in the `course/R_packages/` directory - All students can use them (read-only) - No need to rebuild the image
|
||||||
- Installs packages in the `course/R_packages/` directory
|
|
||||||
- All students can use them (read-only)
|
|
||||||
- No need to rebuild the image
|
|
||||||
|
|
||||||
**From admin notebook:**
|
**From admin notebook:**
|
||||||
|
|
||||||
Login as `admin` and create an R notebook:
|
Login as `admin` and create an R notebook:
|
||||||
|
|
||||||
```r
|
``` r
|
||||||
# Install packages in course/R_packages (admin only, available to all students)
|
# Install packages in course/R_packages (admin only, available to all students)
|
||||||
course_lib <- "/home/jovyan/work/course/R_packages"
|
course_lib <- "/home/jovyan/work/course/R_packages"
|
||||||
dir.create(course_lib, recursive = TRUE, showWarnings = FALSE)
|
dir.create(course_lib, recursive = TRUE, showWarnings = FALSE)
|
||||||
@@ -176,7 +164,7 @@ Note: Admin account has write access to the course directory.
|
|||||||
|
|
||||||
Students can install packages in their personal `work/R_packages/`:
|
Students can install packages in their personal `work/R_packages/`:
|
||||||
|
|
||||||
```r
|
``` r
|
||||||
# Install in personal library (each student has their own)
|
# Install in personal library (each student has their own)
|
||||||
install.packages(c('mypackage')) # Will install in work/R_packages/
|
install.packages(c('mypackage')) # Will install in work/R_packages/
|
||||||
```
|
```
|
||||||
@@ -184,19 +172,17 @@ install.packages(c('mypackage')) # Will install in work/R_packages/
|
|||||||
### Using R Packages (Students)
|
### Using R Packages (Students)
|
||||||
|
|
||||||
Students simply load packages normally:
|
Students simply load packages normally:
|
||||||
```r
|
|
||||||
|
``` r
|
||||||
library(reshape2) # R checks: 1) work/R_packages/ 2) work/course/R_packages/ 3) system
|
library(reshape2) # R checks: 1) work/R_packages/ 2) work/course/R_packages/ 3) system
|
||||||
library(plotly)
|
library(plotly)
|
||||||
```
|
```
|
||||||
|
|
||||||
R automatically searches in this order:
|
R automatically searches in this order: 1. Personal packages: `/home/jovyan/work/R_packages/` (R_LIBS_USER) 2. Prof packages: `/home/jovyan/work/course/R_packages/` (R_LIBS_SITE) 3. System packages
|
||||||
1. Personal packages: `/home/jovyan/work/R_packages/` (R_LIBS_USER)
|
|
||||||
2. Prof packages: `/home/jovyan/work/course/R_packages/` (R_LIBS_SITE)
|
|
||||||
3. System packages
|
|
||||||
|
|
||||||
### List Available Packages
|
### List Available Packages
|
||||||
|
|
||||||
```r
|
``` r
|
||||||
# List all available packages (personal + course + system)
|
# List all available packages (personal + course + system)
|
||||||
installed.packages()[,"Package"]
|
installed.packages()[,"Package"]
|
||||||
|
|
||||||
@@ -211,7 +197,7 @@ list.files("/home/jovyan/work/course/R_packages")
|
|||||||
|
|
||||||
To put files in the `course/` directory (accessible read-only):
|
To put files in the `course/` directory (accessible read-only):
|
||||||
|
|
||||||
```bash
|
``` bash
|
||||||
# Create a temporary directory
|
# Create a temporary directory
|
||||||
mkdir -p ~/jupyterhub-tp/course-files
|
mkdir -p ~/jupyterhub-tp/course-files
|
||||||
|
|
||||||
@@ -230,7 +216,7 @@ docker run --rm \
|
|||||||
|
|
||||||
Students can collaborate via the `shared/` directory:
|
Students can collaborate via the `shared/` directory:
|
||||||
|
|
||||||
```python
|
``` python
|
||||||
# In a notebook, to read a shared file
|
# In a notebook, to read a shared file
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
df = pd.read_csv('/home/jovyan/work/shared/group_data.csv')
|
df = pd.read_csv('/home/jovyan/work/shared/group_data.csv')
|
||||||
@@ -241,7 +227,7 @@ df.to_csv('/home/jovyan/work/shared/alice_results.csv')
|
|||||||
|
|
||||||
### Retrieve Student Work
|
### Retrieve Student Work
|
||||||
|
|
||||||
```bash
|
``` bash
|
||||||
# List user volumes
|
# List user volumes
|
||||||
docker volume ls | grep jupyterhub-user
|
docker volume ls | grep jupyterhub-user
|
||||||
|
|
||||||
@@ -261,14 +247,18 @@ docker run --rm \
|
|||||||
## User Management
|
## User Management
|
||||||
|
|
||||||
### Option 1: Predefined User List
|
### Option 1: Predefined User List
|
||||||
|
|
||||||
In `jupyterhub_config.py`, uncomment and modify:
|
In `jupyterhub_config.py`, uncomment and modify:
|
||||||
```python
|
|
||||||
|
``` python
|
||||||
c.Authenticator.allowed_users = {'student1', 'student2', 'student3'}
|
c.Authenticator.allowed_users = {'student1', 'student2', 'student3'}
|
||||||
```
|
```
|
||||||
|
|
||||||
### Option 2: Allow Everyone (for testing)
|
### Option 2: Allow Everyone (for testing)
|
||||||
|
|
||||||
By default, the configuration allows any user:
|
By default, the configuration allows any user:
|
||||||
```python
|
|
||||||
|
``` python
|
||||||
c.Authenticator.allow_all = True
|
c.Authenticator.allow_all = True
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -276,75 +266,78 @@ c.Authenticator.allow_all = True
|
|||||||
|
|
||||||
## Kernel Verification
|
## Kernel Verification
|
||||||
|
|
||||||
Once logged in, create a new notebook and verify you have access to:
|
Once logged in, create a new notebook and verify you have access to: - **Python 3** (default kernel) - **R** (R kernel) - **Bash** (bash kernel)
|
||||||
- **Python 3** (default kernel)
|
|
||||||
- **R** (R kernel)
|
|
||||||
- **Bash** (bash kernel)
|
|
||||||
|
|
||||||
## Customization for Your Labs
|
## Customization for Your Labs
|
||||||
|
|
||||||
### Add Additional R Packages
|
### Add Additional R Packages
|
||||||
|
|
||||||
Modify the `Dockerfile` (before `USER ${NB_UID}`):
|
Modify the `Dockerfile` (before `USER ${NB_UID}`):
|
||||||
```dockerfile
|
|
||||||
|
``` dockerfile
|
||||||
RUN R -e "install.packages(c('your_package'), repos='http://cran.rstudio.com/')"
|
RUN R -e "install.packages(c('your_package'), repos='http://cran.rstudio.com/')"
|
||||||
```
|
```
|
||||||
|
|
||||||
Then rebuild:
|
Then rebuild:
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
docker build -t jupyterhub-student:latest -f Dockerfile .
|
docker build -t jupyterhub-student:latest -f Dockerfile .
|
||||||
docker-compose restart jupyterhub
|
docker-compose restart jupyterhub
|
||||||
```
|
```
|
||||||
|
|
||||||
### Add Python Packages
|
### Add Python Packages
|
||||||
|
|
||||||
Add to the `Dockerfile` (before `USER ${NB_UID}`):
|
Add to the `Dockerfile` (before `USER ${NB_UID}`):
|
||||||
```dockerfile
|
|
||||||
|
``` dockerfile
|
||||||
RUN pip install numpy pandas matplotlib seaborn
|
RUN pip install numpy pandas matplotlib seaborn
|
||||||
```
|
```
|
||||||
|
|
||||||
### Distribute Files to Students
|
### Distribute Files to Students
|
||||||
|
|
||||||
Create a `files_lab/` directory and add to the `Dockerfile`:
|
Create a `files_lab/` directory and add to the `Dockerfile`:
|
||||||
```dockerfile
|
|
||||||
|
``` dockerfile
|
||||||
COPY files_lab/ /home/${NB_USER}/lab/
|
COPY files_lab/ /home/${NB_USER}/lab/
|
||||||
RUN chown -R ${NB_UID}:${NB_GID} /home/${NB_USER}/lab
|
RUN chown -R ${NB_UID}:${NB_GID} /home/${NB_USER}/lab
|
||||||
```
|
```
|
||||||
|
|
||||||
### Change Port (if 8000 is occupied)
|
### Change Port (if 8000 is occupied)
|
||||||
|
|
||||||
Modify in `docker-compose.yml`:
|
Modify in `docker-compose.yml`:
|
||||||
```yaml
|
|
||||||
|
``` yaml
|
||||||
ports:
|
ports:
|
||||||
- "8001:8000" # Accessible on localhost:8001
|
- "8001:8000" # Accessible on localhost:8001
|
||||||
```
|
```
|
||||||
|
|
||||||
## Advantages of This Approach
|
## Advantages of This Approach
|
||||||
|
|
||||||
✅ **Everything in Docker**: No need to install Python/JupyterHub on your Mac
|
✅ **Everything in Docker**: No need to install Python/JupyterHub on your computer\
|
||||||
✅ **Portable**: Easy to deploy on another Mac or server
|
✅ **Portable**: Easy to deploy on another server\
|
||||||
✅ **Isolated**: No pollution of your system environment
|
✅ **Isolated**: No pollution of your system environment\
|
||||||
✅ **Easy to Clean**: A simple `docker-compose down` is enough
|
✅ **Easy to Clean**: A simple `docker-compose down` is enough\
|
||||||
✅ **Reproducible**: Students will have exactly the same environment
|
✅ **Reproducible**: Students will have exactly the same environment
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
**Error "Cannot connect to Docker daemon"**:
|
**Error "Cannot connect to Docker daemon"**: - Check that OrbStack is running - Verify the socket exists: `ls -la /var/run/docker.sock`
|
||||||
- Check that OrbStack is running
|
|
||||||
- Verify the socket exists: `ls -la /var/run/docker.sock`
|
|
||||||
|
|
||||||
**Student containers don't start**:
|
**Student containers don't start**: - Check logs: `docker-compose logs jupyterhub` - Verify student image exists: `docker images | grep jupyterhub-student`
|
||||||
- Check logs: `docker-compose logs jupyterhub`
|
|
||||||
- Verify student image exists: `docker images | grep jupyterhub-student`
|
|
||||||
|
|
||||||
**Port 8000 already in use**:
|
**Port 8000 already in use**: - Change port in `docker-compose.yml`
|
||||||
- Change port in `docker-compose.yml`
|
|
||||||
|
|
||||||
**After config modification, changes are not applied**:
|
**After config modification, changes are not applied**:
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
docker-compose restart jupyterhub
|
docker-compose restart jupyterhub
|
||||||
```
|
```
|
||||||
|
|
||||||
**I want to start from scratch**:
|
**I want to start from scratch**:
|
||||||
```bash
|
|
||||||
|
``` bash
|
||||||
docker-compose down -v
|
docker-compose down -v
|
||||||
docker rmi jupyterhub-hub jupyterhub-student
|
docker rmi jupyterhub-hub jupyterhub-student
|
||||||
# Then rebuild everything
|
# Then rebuild everything
|
||||||
./start-jupyterhub.sh
|
./start-jupyterhub.sh
|
||||||
```
|
```
|
||||||
Reference in New Issue
Block a user