api-media
Vue d'ensemble
Le service api-media gère le stockage, l'upload et la gestion des médias de la plateforme Roomee. Il utilise Google Cloud Storage (GCP) pour le stockage des fichiers et offre des fonctionnalités avancées comme la génération de thumbnails et la conversion de formats.
| Caractéristique | Détail |
|---|---|
| Port | 3003 |
| Stack | JavaScript (Legacy), Express.js, GCP Storage |
| Base de données | MongoDB (minimal - legacy notifications schema) |
| Architecture | Legacy (JS + GCP SDK) |
| Complexité | ⭐⭐⭐ |
Responsabilités
graph TB
MEDIA[api-media]
subgraph "Fonctionnalités"
UPLOAD[Upload Fichiers]
PRESIGNED[Presigned URLs]
THUMBNAIL[Génération Thumbnails]
CONVERSION[Conversion Formats]
QUOTA[Gestion Quotas]
EDITOR[Editor.js Integration]
end
MEDIA --> UPLOAD
MEDIA --> PRESIGNED
MEDIA --> THUMBNAIL
MEDIA --> CONVERSION
MEDIA --> QUOTA
MEDIA --> EDITOR
subgraph "Stockage"
GCP[Google Cloud Storage]
BUCKETS[Buckets par Écosystème]
end
UPLOAD --> GCP
PRESIGNED --> GCP
THUMBNAIL --> GCP
CONVERSION --> BUCKETSStructure du Projet
api-media/
├── app.js
├── loader.js
├── controllers/
│ ├── media.controller.js # Upload et gestion médias
│ ├── storage.controller.js # Gestion quotas
│ └── editor.controller.js # Editor.js integration
├── routes/
│ ├── media.routes.js
│ ├── storage.routes.js
│ └── editor.routes.js
├── services/
│ ├── gcp.service.js # Google Cloud Storage
│ ├── thumbnail.service.js # Sharp pour thumbnails
│ ├── conversion.service.js # Conversion de formats
│ └── quota.service.js # Gestion des quotas
├── middlewares/
│ ├── auth.middleware.js
│ ├── upload.middleware.js # Multer configuration
│ └── quota.middleware.js # Vérification quota
├── utils/
│ ├── ApiError.js
│ ├── catchAsync.js
│ └── fileHelper.js # Helpers fichiers
├── amqp/
│ └── storageConsumer.js # Consumer storage.created
├── config/
│ └── config.js
└── __tests/Architecture d'Upload
Upload Direct vers GCP (Presigned URLs)
sequenceDiagram
participant Client
participant API as api-media
participant GCP as Google Cloud Storage
Client->>API: POST /media/generate-presigned-url
API->>GCP: Generate signed URL (15min TTL)
GCP-->>API: Signed URL
API-->>Client: { url, metadata }
Client->>GCP: PUT (direct upload)
GCP-->>Client: Upload success
Client->>API: POST /media/upload-complete
API->>GCP: Get file metadata
API->>API: Generate thumbnail
API->>GCP: Upload thumbnail
API-->>Client: { fileUrl, thumbnailUrl }Avantages :
- ✅ Pas de transit par le serveur (économie bande passante)
- ✅ Upload direct client → GCP (meilleure performance)
- ✅ Scalabilité maximale
- ✅ Réduction de la charge serveur
Routes API
📤 Upload Médias (/media)
POST /media/generate-presigned-url
Générer une URL signée pour upload direct vers GCP.
Headers :
Authorization: Bearer eyJhbGciOiJI...Request :
{
"filename": "photo-hotel.jpg",
"contentType": "image/jpeg",
"size": 2048576,
"ecosystemId": "65f1111111111111111111111",
"folder": "photos"
}Response :
{
"uploadUrl": "https://storage.googleapis.com/roomee-storage/...",
"fileUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/photo-hotel.jpg",
"expiresAt": "2025-01-21T10:15:00.000Z",
"metadata": {
"filename": "photo-hotel.jpg",
"contentType": "image/jpeg",
"size": 2048576,
"uploadId": "upload-abc123"
}
}Logique :
const { Storage } = require('@google-cloud/storage')
const storage = new Storage({
projectId: process.env.GCP_PROJECT_ID,
keyFilename: process.env.GCP_KEY_FILE
})
const bucket = storage.bucket(process.env.GCP_BUCKET_NAME)
const filePath = `ecosystems/${ecosystemId}/${folder}/${filename}`
const [url] = await bucket.file(filePath).getSignedUrl({
version: 'v4',
action: 'write',
expires: Date.now() + 15 * 60 * 1000, // 15 minutes
contentType: contentType
})
return { uploadUrl: url, fileUrl: `https://.../${filePath}` }POST /media/upload-complete
Confirmer la fin d'upload et générer le thumbnail.
Request :
{
"uploadId": "upload-abc123",
"fileUrl": "https://storage.googleapis.com/roomee-storage/...",
"generateThumbnail": true
}Response :
{
"fileUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/photo-hotel.jpg",
"thumbnailUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/thumbnails/photo-hotel-thumb.jpg",
"metadata": {
"size": 2048576,
"width": 1920,
"height": 1080,
"format": "jpeg"
}
}Logique :
- Vérifier que le fichier existe sur GCP
- Récupérer les métadonnées du fichier
- Si image, générer le thumbnail (Sharp)
- Uploader le thumbnail vers GCP
- Mettre à jour le quota utilisé
POST /media/upload/
Upload direct via le serveur (fallback).
Headers :
Authorization: Bearer eyJhbGciOiJI...
Content-Type: multipart/form-dataFormData :
file: Fichier à uploaderecosystemId: ID de l'écosystèmefolder: Dossier destination (optionnel)
Response :
{
"fileUrl": "https://storage.googleapis.com/...",
"thumbnailUrl": "https://storage.googleapis.com/...",
"metadata": {
"filename": "photo-hotel.jpg",
"size": 2048576,
"contentType": "image/jpeg"
}
}Limites :
- Taille max : 50 MB
- Formats supportés : Images (JPEG, PNG, WebP, GIF), Vidéos (MP4, WebM), Documents (PDF)
POST /media/upload-chunk
Upload par chunks (pour mobile et gros fichiers).
Request :
{
"chunkIndex": 0,
"totalChunks": 5,
"uploadId": "upload-abc123",
"chunk": "<base64_data>"
}Response :
{
"uploadId": "upload-abc123",
"chunkIndex": 0,
"received": true,
"progress": 20
}Logique :
- Stocker le chunk temporairement
- Quand tous les chunks sont reçus, reconstituer le fichier
- Uploader vers GCP
- Nettoyer les chunks temporaires
DELETE /media/delete/:fileUrl
Supprimer un fichier de GCP.
Response :
{
"message": "File deleted successfully",
"fileUrl": "https://storage.googleapis.com/..."
}GET /media/download/:ecosystemId/:filename
Télécharger un fichier (avec URL signée temporaire).
Response :
{
"downloadUrl": "https://storage.googleapis.com/...",
"expiresAt": "2025-01-21T11:00:00.000Z"
}🖼️ Thumbnails (/media/thumbnail)
POST /media/thumbnail/generate
Générer un thumbnail pour une image.
Request :
{
"fileUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
"width": 300,
"height": 300,
"quality": 80
}Response :
{
"thumbnailUrl": "https://storage.googleapis.com/roomee-storage/thumbnails/photo-thumb.jpg",
"width": 300,
"height": 300,
"size": 45678
}Logique (Sharp) :
const sharp = require('sharp')
// Télécharger l'image depuis GCP
const imageBuffer = await downloadFromGCP(fileUrl)
// Générer le thumbnail
const thumbnailBuffer = await sharp(imageBuffer)
.resize(width, height, {
fit: 'cover',
position: 'center'
})
.jpeg({ quality })
.toBuffer()
// Uploader le thumbnail vers GCP
const thumbnailUrl = await uploadToGCP(thumbnailBuffer, thumbnailPath)
return { thumbnailUrl }POST /media/thumbnail/batch
Générer des thumbnails pour plusieurs images.
🔄 Conversion de Formats (/media/convert)
POST /media/convert/to-webp
Convertir une image en WebP (format optimisé).
Request :
{
"fileUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
"quality": 85
}Response :
{
"originalUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
"convertedUrl": "https://storage.googleapis.com/roomee-storage/photo.webp",
"originalSize": 2048576,
"convertedSize": 512000,
"savings": "75%"
}POST /media/convert/to-jpeg
Convertir une image en JPEG.
📦 Gestion des Quotas (/storage)
POST /storage/create
Créer un quota de stockage pour un écosystème (appelé par api-hotel via AMQP).
Request :
{
"ecosystemId": "65f1111111111111111111111",
"maxSize": 20000,
"unit": "MO"
}Response :
{
"ecosystemId": "65f1111111111111111111111",
"maxSize": 20000,
"unit": "MO",
"used": 0,
"available": 20000,
"percentage": 0
}GET /storage/get/:ecosystemId
Récupérer le quota d'un écosystème.
Response :
{
"ecosystemId": "65f1111111111111111111111",
"maxSize": 20000,
"used": 5432,
"available": 14568,
"percentage": 27.16,
"files": [
{
"path": "ecosystems/65f.../photos/photo1.jpg",
"size": 2048,
"createdAt": "2025-01-20T10:00:00.000Z"
}
]
}PATCH /storage/update/:ecosystemId
Mettre à jour le quota (augmentation/diminution).
✏️ Editor.js Integration (/editor)
POST /editor/upload-by-url
Uploader une image via URL (pour Editor.js).
Request :
{
"url": "https://example.com/image.jpg",
"ecosystemId": "65f1111111111111111111111"
}Response :
{
"success": 1,
"file": {
"url": "https://storage.googleapis.com/roomee-storage/...",
"width": 1920,
"height": 1080
}
}POST /editor/upload-by-file
Uploader une image via fichier (drag & drop Editor.js).
FormData :
image: Fichier imageecosystemId: ID écosystème
Response (Format Editor.js) :
{
"success": 1,
"file": {
"url": "https://storage.googleapis.com/..."
}
}Services
GCP Service
// services/gcp.service.js
const { Storage } = require('@google-cloud/storage')
class GCPService {
constructor() {
this.storage = new Storage({
projectId: process.env.GCP_PROJECT_ID,
keyFilename: process.env.GCP_KEY_FILE
})
this.bucket = this.storage.bucket(process.env.GCP_BUCKET_NAME)
}
async uploadFile(filePath, fileBuffer, contentType) {
const file = this.bucket.file(filePath)
await file.save(fileBuffer, {
contentType: contentType,
metadata: {
cacheControl: 'public, max-age=31536000'
}
})
return file.publicUrl()
}
async deleteFile(filePath) {
const file = this.bucket.file(filePath)
await file.delete()
}
async getSignedUrl(filePath, action, expiresIn) {
const file = this.bucket.file(filePath)
const [url] = await file.getSignedUrl({
version: 'v4',
action: action, // 'read' or 'write'
expires: Date.now() + expiresIn
})
return url
}
async getFileMetadata(filePath) {
const file = this.bucket.file(filePath)
const [metadata] = await file.getMetadata()
return {
size: metadata.size,
contentType: metadata.contentType,
createdAt: metadata.timeCreated,
updatedAt: metadata.updated
}
}
async listFiles(prefix) {
const [files] = await this.bucket.getFiles({ prefix })
return files.map(file => ({
name: file.name,
size: file.metadata.size,
contentType: file.metadata.contentType,
createdAt: file.metadata.timeCreated
}))
}
}Thumbnail Service
// services/thumbnail.service.js
const sharp = require('sharp')
class ThumbnailService {
async generateThumbnail(imageBuffer, options = {}) {
const {
width = 300,
height = 300,
quality = 80,
format = 'jpeg'
} = options
const thumbnail = await sharp(imageBuffer)
.resize(width, height, {
fit: 'cover',
position: 'center'
})
.toFormat(format, { quality })
.toBuffer()
return thumbnail
}
async generateMultipleSizes(imageBuffer, sizes) {
const thumbnails = {}
for (const [name, { width, height }] of Object.entries(sizes)) {
thumbnails[name] = await this.generateThumbnail(imageBuffer, {
width,
height
})
}
return thumbnails
}
async getImageMetadata(imageBuffer) {
const metadata = await sharp(imageBuffer).metadata()
return {
width: metadata.width,
height: metadata.height,
format: metadata.format,
size: metadata.size
}
}
}
// Tailles prédéfinies
const THUMBNAIL_SIZES = {
small: { width: 150, height: 150 },
medium: { width: 300, height: 300 },
large: { width: 600, height: 600 }
}Conversion Service
// services/conversion.service.js
const sharp = require('sharp')
class ConversionService {
async convertToWebP(imageBuffer, quality = 85) {
return await sharp(imageBuffer)
.webp({ quality })
.toBuffer()
}
async convertToJPEG(imageBuffer, quality = 90) {
return await sharp(imageBuffer)
.jpeg({ quality })
.toBuffer()
}
async convertToPNG(imageBuffer) {
return await sharp(imageBuffer)
.png()
.toBuffer()
}
async optimizeImage(imageBuffer, format) {
const options = {
jpeg: { quality: 85, progressive: true },
png: { compressionLevel: 9 },
webp: { quality: 85 }
}
return await sharp(imageBuffer)
.toFormat(format, options[format])
.toBuffer()
}
}Quota Service
// services/quota.service.js
class QuotaService {
async getQuota(ecosystemId) {
// Récupérer le quota depuis la base
const quota = await prisma.storage.findUnique({
where: { ecosystemId }
})
if (!quota) {
throw new ApiError(404, 'Quota not found')
}
// Calculer l'utilisation réelle depuis GCP
const files = await gcpService.listFiles(`ecosystems/${ecosystemId}`)
const used = files.reduce((sum, file) => sum + file.size, 0)
return {
ecosystemId,
maxSize: quota.maxSize,
used: Math.round(used / (1024 * 1024)), // MB
available: quota.maxSize - Math.round(used / (1024 * 1024)),
percentage: (used / (quota.maxSize * 1024 * 1024)) * 100,
files: files.length
}
}
async checkQuota(ecosystemId, fileSize) {
const quota = await this.getQuota(ecosystemId)
const fileSizeMB = fileSize / (1024 * 1024)
if (quota.available < fileSizeMB) {
throw new ApiError(413, 'Storage quota exceeded')
}
return true
}
async updateUsage(ecosystemId, delta) {
await prisma.storage.update({
where: { ecosystemId },
data: {
used: {
increment: delta
}
}
})
}
}Middlewares
Upload Middleware (Multer)
// middlewares/upload.middleware.js
const multer = require('multer')
// Stockage en mémoire (buffer)
const storage = multer.memoryStorage()
const upload = multer({
storage: storage,
limits: {
fileSize: 50 * 1024 * 1024 // 50 MB
},
fileFilter: (req, file, cb) => {
const allowedTypes = [
'image/jpeg',
'image/png',
'image/gif',
'image/webp',
'video/mp4',
'video/webm',
'application/pdf'
]
if (allowedTypes.includes(file.mimetype)) {
cb(null, true)
} else {
cb(new ApiError(400, 'Invalid file type'))
}
}
})
module.exports = {
uploadSingle: upload.single('file'),
uploadMultiple: upload.array('files', 10),
uploadFields: upload.fields([
{ name: 'image', maxCount: 1 },
{ name: 'thumbnail', maxCount: 1 }
])
}Quota Middleware
// middlewares/quota.middleware.js
const checkQuota = async (req, res, next) => {
const { ecosystemId } = req.body
const fileSize = req.file ? req.file.size : req.body.size
try {
await quotaService.checkQuota(ecosystemId, fileSize)
next()
} catch (error) {
res.status(413).json({
error: 'Storage quota exceeded',
quota: await quotaService.getQuota(ecosystemId)
})
}
}Événements AMQP
Événements Consommés
| Routing Key | Payload | Action |
|---|---|---|
roomee.storage.media | { ecosystemId, max_size, unit } | Créer quota storage |
Consumer :
// amqp/storageConsumer.js
amqpServer.subscribe('roomee.storage.media', async (message) => {
const { ecosystemId, payload } = message
await prisma.storage.create({
data: {
ecosystemId: ecosystemId,
maxSize: payload.max_size,
unit: payload.unit,
used: 0
}
})
logger.info('Storage quota created', { ecosystemId, maxSize: payload.max_size })
})Configuration
Variables d'Environnement
# Server
PORT=3003
NODE_ENV=development
# Database (minimal)
DATABASE_URL=mongodb://localhost:27017/roomee_media
# JWT
JWT_ACCESS_SECRET=your-jwt-secret
# Google Cloud Storage
GCP_PROJECT_ID=your-gcp-project-id
GCP_BUCKET_NAME=roomee-storage
GCP_KEY_FILE=./config/gcp-service-account-key.json
# Upload Limits
MAX_FILE_SIZE=52428800
MAX_CHUNK_SIZE=1048576
# AMQP
AMQP_GATEWAY_URL=amqp://localhost:5672
AMQP_EXCHANGE_NAME=roomee_events
AMQP_ROUTING_KEY_BASE=roomee.storageFichier GCP Service Account
{
"type": "service_account",
"project_id": "your-project",
"private_key_id": "...",
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
"client_email": "storage@your-project.iam.gserviceaccount.com",
"client_id": "...",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token"
}Organisation GCP Bucket
roomee-storage/
├── ecosystems/
│ ├── {ecosystemId}/
│ │ ├── photos/
│ │ │ ├── photo1.jpg
│ │ │ └── thumbnails/
│ │ │ └── photo1-thumb.jpg
│ │ ├── documents/
│ │ │ └── menu.pdf
│ │ ├── videos/
│ │ │ └── presentation.mp4
│ │ └── avatars/
│ │ └── member-123.jpg
├── qr-codes/
│ └── hotel-{hotelId}.png
└── temp/
└── chunks/
└── upload-{uploadId}/Tests
// __tests__/media.test.js
describe('MediaController', () => {
describe('generatePresignedUrl', () => {
it('should generate a presigned URL', async () => {
const response = await request(app)
.post('/media/generate-presigned-url')
.set('Authorization', `Bearer ${token}`)
.send({
filename: 'test.jpg',
contentType: 'image/jpeg',
size: 1024000,
ecosystemId: ecosystemId,
folder: 'photos'
})
.expect(200)
expect(response.body).toHaveProperty('uploadUrl')
expect(response.body).toHaveProperty('fileUrl')
expect(response.body).toHaveProperty('expiresAt')
})
})
describe('uploadFile', () => {
it('should upload a file to GCP', async () => {
const response = await request(app)
.post('/media/upload')
.set('Authorization', `Bearer ${token}`)
.attach('file', './test-files/image.jpg')
.field('ecosystemId', ecosystemId)
.field('folder', 'photos')
.expect(200)
expect(response.body).toHaveProperty('fileUrl')
expect(response.body).toHaveProperty('thumbnailUrl')
})
})
})Bonnes Pratiques
1. Toujours vérifier le quota avant upload
await quotaService.checkQuota(ecosystemId, fileSize)2. Générer des thumbnails automatiquement pour les images
if (contentType.startsWith('image/')) {
const thumbnail = await thumbnailService.generateThumbnail(fileBuffer)
const thumbnailUrl = await gcpService.uploadFile(thumbnailPath, thumbnail)
}3. Optimiser les images avant upload
const optimized = await conversionService.optimizeImage(fileBuffer, 'webp')4. Nettoyer les fichiers temporaires
// Après upload par chunks
await cleanupTempFiles(uploadId)5. Utiliser des presigned URLs pour les uploads
// Meilleur pour la performance
const { uploadUrl } = await generatePresignedUrl()
// Client upload directement vers GCPLimitations et Améliorations Futures
Limitations Actuelles
- ❌ Pas de CDN devant GCP Storage
- ❌ Pas de compression vidéo
- ❌ Pas de watermarking
- ❌ Pas de détection de contenu inapproprié
Roadmap
- [ ] Intégration CDN (Cloudflare/CloudFront)
- [ ] Compression vidéo (FFmpeg)
- [ ] Watermarking automatique
- [ ] AI Content Moderation (Google Vision API)
- [ ] Multi-région GCP
- [ ] Support S3 (alternative à GCP)
- [ ] Galleries et albums
- [ ] Métadonnées EXIF preservation