MetabarcodingSchool/OBIJupyterHub
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Image multistage

Browse Source
This commit is contained in:
Eric Coissac
2025-10-15 14:08:52 +02:00
parent 60130d6b42
commit 64cb75e40a
5 changed files with 86 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
Dockerfile
View File

@@ -1,47 +1,49 @@
FROM jupyter/base-notebook:latest # ---------- Stage 1 : builder ----------
FROM jupyter/base-notebook:latest AS builder
USER root USER root
# Install R and system dependencies # Install system dependencies for R, build tools and Go/Rust
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
r-base \ r-base r-base-dev \
r-base-dev \ libcurl4-openssl-dev libssl-dev libxml2-dev \
libcurl4-openssl-dev \ build-essential git curl \
libssl-dev \
libxml2-dev \
git \
build-essential \
curl \
&& apt-get clean && rm -rf /var/lib/apt/lists/* && apt-get clean && rm -rf /var/lib/apt/lists/*
# Install R kernel for Jupyter (as root) # Install R kernel + useful packages
RUN R -e "install.packages('IRkernel', repos='http://cran.rstudio.com/')" && \ RUN R -e "install.packages('IRkernel', repos='http://cran.rstudio.com/')" && \
R -e "IRkernel::installspec(user = FALSE)" R -e "IRkernel::installspec(user = FALSE)" && \
R -e "install.packages(c('tidyverse','vegan','ade4'), repos='http://cran.rstudio.com/')"
# Install some useful R packages for labs # Install bash kernel
RUN R -e "install.packages(c('tidyverse', 'vegan', 'ade4'), repos='http://cran.rstudio.com/')" RUN pip install bash_kernel && python -m bash_kernel.install --sys-prefix
# Install bash kernel (as root also) # Install obitools4
RUN pip install bash_kernel && \
python -m bash_kernel.install --sys-prefix
# Install obitools4 (written in Go)
RUN curl -L https://raw.githubusercontent.com/metabarcoding/obitools4/master/install_obitools.sh | bash RUN curl -L https://raw.githubusercontent.com/metabarcoding/obitools4/master/install_obitools.sh | bash
# Install csvkit # Install csvkit
RUN pip install csvkit RUN pip install csvkit
# Install csvlens # Install csvlens via Rust
RUN curl https://sh.rustup.rs -sSf | bash -s -- -y \ RUN curl https://sh.rustup.rs -sSf | bash -s -- -y && \
&& . $HOME/.cargo/env \ . $HOME/.cargo/env && \
&& cargo install csvlens \ cargo install csvlens
&& mv $HOME/.cargo/bin/csvlens /usr/local/bin/ \
&& rm -rf $HOME/.cargo $HOME/.rustup
# Create necessary directories with proper permissions RUN apt-get update && apt-get install -y ruby ruby-dev build-essential \
&& gem install youplot
# Copy csvlens to /usr/local/bin for final use
RUN cp $HOME/.cargo/bin/csvlens /usr/local/bin/
# Set permissions for Jupyter user
RUN mkdir -p /home/${NB_USER}/.local/share/jupyter && \ RUN mkdir -p /home/${NB_USER}/.local/share/jupyter && \
chown -R ${NB_UID}:${NB_GID} /home/${NB_USER} chown -R ${NB_UID}:${NB_GID} /home/${NB_USER}
USER ${NB_UID} # Switch back to Jupyter user
USER ${NB_UID}:${NB_GID}
WORKDIR /home/${NB_USER}/work
WORKDIR /home/${NB_USER} # Environment variables
ENV PATH="/home/${NB_USER}/work/course/bin:${PATH}"
ENV R_LIBS_USER="/home/${NB_USER}/work/R_packages"
ENV R_LIBS_SITE="/home/${NB_USER}/work/course/R_packages:/usr/local/lib/R/site-library:/usr/lib/R/site-library"

60
Readme.md
View File

@@ -47,7 +47,7 @@ chmod +x start-jupyterhub.sh
### 5. Access JupyterHub ### 5. Access JupyterHub
Open your browser and go to: **http://localhost:8000** Open your browser and go to: **http://localhost:8888**
You can log in with any username and password: `metabar2025` You can log in with any username and password: `metabar2025`
@@ -109,11 +109,24 @@ docker volume prune -f
### Directory Structure for Each Student ### Directory Structure for Each Student
Each student will see these directories in their JupyterLab: Each student will see this directory structure in their JupyterLab (everything under `work/` is persistent):
- **`work/`** : Personal workspace (persistent, private) ```
- **`shared/`** : Shared workspace between all students (read/write) work/ # Personal workspace root (persistent)
- **`course/`** : Course files (read-only, you deposit files) ├── [student files] # Their own files and notebooks
- **`course/R_packages/`** : Shared R packages (read-only for students, only admin can install) ├── R_packages/ # Personal R packages (writable by student)
├── shared/ # Shared workspace (read/write, shared with all)
└── course/ # Course files (read-only, managed by admin)
├── R_packages/ # Shared R packages (read-only, installed by prof)
├── bin/ # Shared executables (in PATH)
└── [course materials] # Your course files
```
**R Package Priority:**
1. R checks `work/R_packages/` first (personal, writable)
2. Then `work/course/R_packages/` (shared, read-only, installed by prof)
3. Then system libraries
**Important:** Everything is under `work/`, so all student files are automatically saved in their persistent volume.
### User Accounts ### User Accounts
@@ -148,8 +161,8 @@ This script:
Login as `admin` and create an R notebook: Login as `admin` and create an R notebook:
```r ```r
# Install packages in course directory (admin only) # Install packages in course/R_packages (admin only, available to all students)
course_lib <- "/home/jovyan/course/R_packages" course_lib <- "/home/jovyan/work/course/R_packages"
dir.create(course_lib, recursive = TRUE, showWarnings = FALSE) dir.create(course_lib, recursive = TRUE, showWarnings = FALSE)
install.packages(c('reshape2', 'plotly', 'knitr'), install.packages(c('reshape2', 'plotly', 'knitr'),
@@ -159,24 +172,39 @@ install.packages(c('reshape2', 'plotly', 'knitr'),
Note: Admin account has write access to the course directory. Note: Admin account has write access to the course directory.
**Students can also install their own packages:**
Students can install packages in their personal `work/R_packages/`:
```r
# Install in personal library (each student has their own)
install.packages(c('mypackage')) # Will install in work/R_packages/
```
### Using R Packages (Students) ### Using R Packages (Students)
Students simply load packages normally: Students simply load packages normally:
```r ```r
library(reshape2) # Loads from course/R_packages/ automatically library(reshape2) # R checks: 1) work/R_packages/ 2) work/course/R_packages/ 3) system
library(plotly) library(plotly)
``` ```
R automatically finds packages in `/home/jovyan/course/R_packages/` thanks to the `R_LIBS_USER` environment variable. R automatically searches in this order:
1. Personal packages: `/home/jovyan/work/R_packages/` (R_LIBS_USER)
2. Prof packages: `/home/jovyan/work/course/R_packages/` (R_LIBS_SITE)
3. System packages
### List Available Packages ### List Available Packages
```r ```r
# List all available packages # List all available packages (personal + course + system)
installed.packages()[,"Package"] installed.packages()[,"Package"]
# Or check course packages specifically # Check personal packages
list.files("/home/jovyan/course/R_packages") list.files("/home/jovyan/work/R_packages")
# Check course packages (installed by prof)
list.files("/home/jovyan/work/course/R_packages")
``` ```
### Deposit Files for Course ### Deposit Files for Course
@@ -205,10 +233,10 @@ Students can collaborate via the `shared/` directory:
```python ```python
# In a notebook, to read a shared file # In a notebook, to read a shared file
import pandas as pd import pandas as pd
df = pd.read_csv('/home/jovyan/shared/group_data.csv') df = pd.read_csv('/home/jovyan/work/shared/group_data.csv')
# To write a shared file # To write a shared file
df.to_csv('/home/jovyan/shared/alice_results.csv') df.to_csv('/home/jovyan/work/shared/alice_results.csv')
``` ```
### Retrieve Student Work ### Retrieve Student Work
@@ -319,4 +347,4 @@ docker-compose down -v
docker rmi jupyterhub-hub jupyterhub-student docker rmi jupyterhub-hub jupyterhub-student
# Then rebuild everything # Then rebuild everything
./start-jupyterhub.sh ./start-jupyterhub.sh
``` ```

8
install_packages.sh
View File

@@ -50,9 +50,9 @@ docker run --rm \
exit 1 exit 1
} }
# Copy to course volume # Copy to course R packages directory
echo "" echo ""
echo -e "${BLUE}💾 Copying to course volume...${NC}" echo -e "${BLUE}💾 Copying to course/R_packages...${NC}"
docker run --rm \ docker run --rm \
-v jupyterhub-course:/target \ -v jupyterhub-course:/target \
-v "${TEMP_DIR}:/source" \ -v "${TEMP_DIR}:/source" \
@@ -65,10 +65,10 @@ rm -rf "${TEMP_DIR}"
echo "" echo ""
echo -e "${GREEN}✅ Installation complete!${NC}" echo -e "${GREEN}✅ Installation complete!${NC}"
echo "" echo ""
echo -e "${BLUE}📦 Installed packages in course/R_packages:${NC}" echo -e "${BLUE}📦 Installed packages in work/course/R_packages:${NC}"
docker run --rm \ docker run --rm \
-v jupyterhub-course:/course \ -v jupyterhub-course:/course \
alpine ls -1 /course/R_packages/ alpine ls -1 /course/R_packages/
echo "" echo ""
echo -e "${YELLOW}ℹ️ Students need to restart their R kernels to use new packages.${NC}" echo -e "${YELLOW}ℹ️ Students need to restart their R kernels to use new packages.${NC}"

39
jupyterhub_config.py
View File

@@ -54,23 +54,6 @@ c.DockerSpawner.volumes = {
c.DockerSpawner.mem_limit = '2G' c.DockerSpawner.mem_limit = '2G'
c.DockerSpawner.cpu_limit = 1.0 c.DockerSpawner.cpu_limit = 1.0
# Environment variables for student containers
c.DockerSpawner.environment = {
# R package library in read-only course directory under work/
'R_LIBS_USER': '/home/jovyan/work/R_packages',
'R_LIBS_SITE': '/home/jovyan/work/course/R_packages',
# Path to R packages in read-only
'PATH': '/home/jovyan/work/course/bin:${PATH}'
}
# Create user R lib directory
#async def create_user_hierarchy(spawner):
# cmd = "mkdir -p /home/jovyan/work/R_packages && chown jovyan:jovyan /home/jovyan/work/R_packages"
# spawner.extra_create_kwargs.update({'command': f"/bin/bash -c '{cmd} && start-notebook.sh'"})
#c.Spawner.pre_spawn_hook = create_user_r_libs
# User configuration - Simple password authentication for lab # User configuration - Simple password authentication for lab
from jupyterhub.auth import DummyAuthenticator from jupyterhub.auth import DummyAuthenticator
@@ -166,26 +149,6 @@ c.DockerSpawner.volumes = {
c.DockerSpawner.mem_limit = '2G' c.DockerSpawner.mem_limit = '2G'
c.DockerSpawner.cpu_limit = 1.0 c.DockerSpawner.cpu_limit = 1.0
# Configuration des utilisateurs - Mot de passe simple pour TP
from jupyterhub.auth import DummyAuthenticator
class SimplePasswordAuthenticator(DummyAuthenticator):
"""Authentificateur simple avec un mot de passe partagé pour tous"""
def check_allowed(self, username, authentication=None):
"""Vérifie si l'utilisateur est autorisé"""
if authentication is None:
return False
# Récupérer le mot de passe depuis la variable d'environnement
expected_password = os.environ.get('JUPYTERHUB_PASSWORD', 'metabar2025')
provided_password = authentication.get('password', '')
# Vérifier le mot de passe
return provided_password == expected_password
c.JupyterHub.authenticator_class = SimplePasswordAuthenticator
# Pour créer une liste d'utilisateurs autorisés, décommentez et modifiez: # Pour créer une liste d'utilisateurs autorisés, décommentez et modifiez:
# c.Authenticator.allowed_users = {'etudiant1', 'etudiant2', 'etudiant3'} # c.Authenticator.allowed_users = {'etudiant1', 'etudiant2', 'etudiant3'}
@@ -200,4 +163,4 @@ c.JupyterHub.bind_url = 'http://0.0.0.0:8000'
# Timeout # Timeout
c.Spawner.start_timeout = 300 c.Spawner.start_timeout = 300
c.Spawner.http_timeout = 120 c.Spawner.http_timeout = 120

9
setup.sh
View File

@@ -72,9 +72,12 @@ if docker ps | grep -q jupyterhub; then
echo " Password: admin2025" echo " Password: admin2025"
echo "" echo ""
echo "📂 Each student will have access to:" echo "📂 Each student will have access to:"
echo " - work/ : personal workspace (everything here is saved)" echo " - work/ : personal workspace (everything saved)"
echo " - work/shared/ : shared workspace" echo " - work/R_packages/ : personal R packages (writable)"
echo " - work/course/ : course files (read-only)" echo " - work/shared/ : shared workspace"
echo " - work/course/ : course files (read-only)"
echo " - work/course/R_packages/ : shared R packages by prof (read-only)"
echo " - work/course/bin/ : shared executables (in PATH)"
echo "" echo ""
echo "🔍 To view logs: docker-compose logs -f jupyterhub" echo "🔍 To view logs: docker-compose logs -f jupyterhub"
echo "🛑 To stop: docker-compose down" echo "🛑 To stop: docker-compose down"