Skip to content

api-media ​

Vue d'ensemble ​

Le service api-media gère le stockage, l'upload et la gestion des médias de la plateforme Roomee. Il utilise Google Cloud Storage (GCP) pour le stockage des fichiers et offre des fonctionnalités avancées comme la génération de thumbnails et la conversion de formats.

CaractéristiqueDétail
Port3003
StackJavaScript (Legacy), Express.js, GCP Storage
Base de donnéesMongoDB (minimal - legacy notifications schema)
ArchitectureLegacy (JS + GCP SDK)
Complexité⭐⭐⭐

Responsabilités ​

mermaid
graph TB
    MEDIA[api-media]

    subgraph "Fonctionnalités"
        UPLOAD[Upload Fichiers]
        PRESIGNED[Presigned URLs]
        THUMBNAIL[Génération Thumbnails]
        CONVERSION[Conversion Formats]
        QUOTA[Gestion Quotas]
        EDITOR[Editor.js Integration]
    end

    MEDIA --> UPLOAD
    MEDIA --> PRESIGNED
    MEDIA --> THUMBNAIL
    MEDIA --> CONVERSION
    MEDIA --> QUOTA
    MEDIA --> EDITOR

    subgraph "Stockage"
        GCP[Google Cloud Storage]
        BUCKETS[Buckets par Écosystème]
    end

    UPLOAD --> GCP
    PRESIGNED --> GCP
    THUMBNAIL --> GCP
    CONVERSION --> BUCKETS

Structure du Projet ​

api-media/
├── app.js
├── loader.js
├── controllers/
│   ├── media.controller.js         # Upload et gestion médias
│   ├── storage.controller.js       # Gestion quotas
│   └── editor.controller.js        # Editor.js integration
├── routes/
│   ├── media.routes.js
│   ├── storage.routes.js
│   └── editor.routes.js
├── services/
│   ├── gcp.service.js              # Google Cloud Storage
│   ├── thumbnail.service.js        # Sharp pour thumbnails
│   ├── conversion.service.js       # Conversion de formats
│   └── quota.service.js            # Gestion des quotas
├── middlewares/
│   ├── auth.middleware.js
│   ├── upload.middleware.js        # Multer configuration
│   └── quota.middleware.js         # Vérification quota
├── utils/
│   ├── ApiError.js
│   ├── catchAsync.js
│   └── fileHelper.js               # Helpers fichiers
├── amqp/
│   └── storageConsumer.js          # Consumer storage.created
├── config/
│   └── config.js
└── __tests/

Architecture d'Upload ​

Upload Direct vers GCP (Presigned URLs) ​

mermaid
sequenceDiagram
    participant Client
    participant API as api-media
    participant GCP as Google Cloud Storage

    Client->>API: POST /media/generate-presigned-url
    API->>GCP: Generate signed URL (15min TTL)
    GCP-->>API: Signed URL
    API-->>Client: { url, metadata }

    Client->>GCP: PUT (direct upload)
    GCP-->>Client: Upload success

    Client->>API: POST /media/upload-complete
    API->>GCP: Get file metadata
    API->>API: Generate thumbnail
    API->>GCP: Upload thumbnail
    API-->>Client: { fileUrl, thumbnailUrl }

Avantages :

  • ✅ Pas de transit par le serveur (économie bande passante)
  • ✅ Upload direct client → GCP (meilleure performance)
  • ✅ Scalabilité maximale
  • ✅ Réduction de la charge serveur

Routes API ​

📤 Upload Médias (/media) ​

POST /media/generate-presigned-url ​

Générer une URL signée pour upload direct vers GCP.

Headers :

Authorization: Bearer eyJhbGciOiJI...

Request :

json
{
  "filename": "photo-hotel.jpg",
  "contentType": "image/jpeg",
  "size": 2048576,
  "ecosystemId": "65f1111111111111111111111",
  "folder": "photos"
}

Response :

json
{
  "uploadUrl": "https://storage.googleapis.com/roomee-storage/...",
  "fileUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/photo-hotel.jpg",
  "expiresAt": "2025-01-21T10:15:00.000Z",
  "metadata": {
    "filename": "photo-hotel.jpg",
    "contentType": "image/jpeg",
    "size": 2048576,
    "uploadId": "upload-abc123"
  }
}

Logique :

javascript
const { Storage } = require('@google-cloud/storage')
const storage = new Storage({
  projectId: process.env.GCP_PROJECT_ID,
  keyFilename: process.env.GCP_KEY_FILE
})

const bucket = storage.bucket(process.env.GCP_BUCKET_NAME)
const filePath = `ecosystems/${ecosystemId}/${folder}/${filename}`

const [url] = await bucket.file(filePath).getSignedUrl({
  version: 'v4',
  action: 'write',
  expires: Date.now() + 15 * 60 * 1000, // 15 minutes
  contentType: contentType
})

return { uploadUrl: url, fileUrl: `https://.../${filePath}` }

POST /media/upload-complete ​

Confirmer la fin d'upload et générer le thumbnail.

Request :

json
{
  "uploadId": "upload-abc123",
  "fileUrl": "https://storage.googleapis.com/roomee-storage/...",
  "generateThumbnail": true
}

Response :

json
{
  "fileUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/photo-hotel.jpg",
  "thumbnailUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/thumbnails/photo-hotel-thumb.jpg",
  "metadata": {
    "size": 2048576,
    "width": 1920,
    "height": 1080,
    "format": "jpeg"
  }
}

Logique :

  1. Vérifier que le fichier existe sur GCP
  2. Récupérer les métadonnées du fichier
  3. Si image, générer le thumbnail (Sharp)
  4. Uploader le thumbnail vers GCP
  5. Mettre à jour le quota utilisé

POST /media/upload/ ​

Upload direct via le serveur (fallback).

Headers :

Authorization: Bearer eyJhbGciOiJI...
Content-Type: multipart/form-data

FormData :

  • file : Fichier à uploader
  • ecosystemId : ID de l'écosystème
  • folder : Dossier destination (optionnel)

Response :

json
{
  "fileUrl": "https://storage.googleapis.com/...",
  "thumbnailUrl": "https://storage.googleapis.com/...",
  "metadata": {
    "filename": "photo-hotel.jpg",
    "size": 2048576,
    "contentType": "image/jpeg"
  }
}

Limites :

  • Taille max : 50 MB
  • Formats supportés : Images (JPEG, PNG, WebP, GIF), Vidéos (MP4, WebM), Documents (PDF)

POST /media/upload-chunk ​

Upload par chunks (pour mobile et gros fichiers).

Request :

json
{
  "chunkIndex": 0,
  "totalChunks": 5,
  "uploadId": "upload-abc123",
  "chunk": "<base64_data>"
}

Response :

json
{
  "uploadId": "upload-abc123",
  "chunkIndex": 0,
  "received": true,
  "progress": 20
}

Logique :

  1. Stocker le chunk temporairement
  2. Quand tous les chunks sont reçus, reconstituer le fichier
  3. Uploader vers GCP
  4. Nettoyer les chunks temporaires

DELETE /media/delete/:fileUrl ​

Supprimer un fichier de GCP.

Response :

json
{
  "message": "File deleted successfully",
  "fileUrl": "https://storage.googleapis.com/..."
}

GET /media/download/:ecosystemId/:filename ​

Télécharger un fichier (avec URL signée temporaire).

Response :

json
{
  "downloadUrl": "https://storage.googleapis.com/...",
  "expiresAt": "2025-01-21T11:00:00.000Z"
}

🖼️ Thumbnails (/media/thumbnail) ​

POST /media/thumbnail/generate ​

Générer un thumbnail pour une image.

Request :

json
{
  "fileUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
  "width": 300,
  "height": 300,
  "quality": 80
}

Response :

json
{
  "thumbnailUrl": "https://storage.googleapis.com/roomee-storage/thumbnails/photo-thumb.jpg",
  "width": 300,
  "height": 300,
  "size": 45678
}

Logique (Sharp) :

javascript
const sharp = require('sharp')

// Télécharger l'image depuis GCP
const imageBuffer = await downloadFromGCP(fileUrl)

// Générer le thumbnail
const thumbnailBuffer = await sharp(imageBuffer)
  .resize(width, height, {
    fit: 'cover',
    position: 'center'
  })
  .jpeg({ quality })
  .toBuffer()

// Uploader le thumbnail vers GCP
const thumbnailUrl = await uploadToGCP(thumbnailBuffer, thumbnailPath)

return { thumbnailUrl }

POST /media/thumbnail/batch ​

Générer des thumbnails pour plusieurs images.


🔄 Conversion de Formats (/media/convert) ​

POST /media/convert/to-webp ​

Convertir une image en WebP (format optimisé).

Request :

json
{
  "fileUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
  "quality": 85
}

Response :

json
{
  "originalUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
  "convertedUrl": "https://storage.googleapis.com/roomee-storage/photo.webp",
  "originalSize": 2048576,
  "convertedSize": 512000,
  "savings": "75%"
}

POST /media/convert/to-jpeg ​

Convertir une image en JPEG.


📦 Gestion des Quotas (/storage) ​

POST /storage/create ​

Créer un quota de stockage pour un écosystème (appelé par api-hotel via AMQP).

Request :

json
{
  "ecosystemId": "65f1111111111111111111111",
  "maxSize": 20000,
  "unit": "MO"
}

Response :

json
{
  "ecosystemId": "65f1111111111111111111111",
  "maxSize": 20000,
  "unit": "MO",
  "used": 0,
  "available": 20000,
  "percentage": 0
}

GET /storage/get/:ecosystemId ​

Récupérer le quota d'un écosystème.

Response :

json
{
  "ecosystemId": "65f1111111111111111111111",
  "maxSize": 20000,
  "used": 5432,
  "available": 14568,
  "percentage": 27.16,
  "files": [
    {
      "path": "ecosystems/65f.../photos/photo1.jpg",
      "size": 2048,
      "createdAt": "2025-01-20T10:00:00.000Z"
    }
  ]
}

PATCH /storage/update/:ecosystemId ​

Mettre à jour le quota (augmentation/diminution).


✏️ Editor.js Integration (/editor) ​

POST /editor/upload-by-url ​

Uploader une image via URL (pour Editor.js).

Request :

json
{
  "url": "https://example.com/image.jpg",
  "ecosystemId": "65f1111111111111111111111"
}

Response :

json
{
  "success": 1,
  "file": {
    "url": "https://storage.googleapis.com/roomee-storage/...",
    "width": 1920,
    "height": 1080
  }
}

POST /editor/upload-by-file ​

Uploader une image via fichier (drag & drop Editor.js).

FormData :

  • image : Fichier image
  • ecosystemId : ID écosystème

Response (Format Editor.js) :

json
{
  "success": 1,
  "file": {
    "url": "https://storage.googleapis.com/..."
  }
}

Services ​

GCP Service ​

javascript
// services/gcp.service.js
const { Storage } = require('@google-cloud/storage')

class GCPService {
  constructor() {
    this.storage = new Storage({
      projectId: process.env.GCP_PROJECT_ID,
      keyFilename: process.env.GCP_KEY_FILE
    })
    this.bucket = this.storage.bucket(process.env.GCP_BUCKET_NAME)
  }

  async uploadFile(filePath, fileBuffer, contentType) {
    const file = this.bucket.file(filePath)

    await file.save(fileBuffer, {
      contentType: contentType,
      metadata: {
        cacheControl: 'public, max-age=31536000'
      }
    })

    return file.publicUrl()
  }

  async deleteFile(filePath) {
    const file = this.bucket.file(filePath)
    await file.delete()
  }

  async getSignedUrl(filePath, action, expiresIn) {
    const file = this.bucket.file(filePath)

    const [url] = await file.getSignedUrl({
      version: 'v4',
      action: action, // 'read' or 'write'
      expires: Date.now() + expiresIn
    })

    return url
  }

  async getFileMetadata(filePath) {
    const file = this.bucket.file(filePath)
    const [metadata] = await file.getMetadata()

    return {
      size: metadata.size,
      contentType: metadata.contentType,
      createdAt: metadata.timeCreated,
      updatedAt: metadata.updated
    }
  }

  async listFiles(prefix) {
    const [files] = await this.bucket.getFiles({ prefix })

    return files.map(file => ({
      name: file.name,
      size: file.metadata.size,
      contentType: file.metadata.contentType,
      createdAt: file.metadata.timeCreated
    }))
  }
}

Thumbnail Service ​

javascript
// services/thumbnail.service.js
const sharp = require('sharp')

class ThumbnailService {
  async generateThumbnail(imageBuffer, options = {}) {
    const {
      width = 300,
      height = 300,
      quality = 80,
      format = 'jpeg'
    } = options

    const thumbnail = await sharp(imageBuffer)
      .resize(width, height, {
        fit: 'cover',
        position: 'center'
      })
      .toFormat(format, { quality })
      .toBuffer()

    return thumbnail
  }

  async generateMultipleSizes(imageBuffer, sizes) {
    const thumbnails = {}

    for (const [name, { width, height }] of Object.entries(sizes)) {
      thumbnails[name] = await this.generateThumbnail(imageBuffer, {
        width,
        height
      })
    }

    return thumbnails
  }

  async getImageMetadata(imageBuffer) {
    const metadata = await sharp(imageBuffer).metadata()

    return {
      width: metadata.width,
      height: metadata.height,
      format: metadata.format,
      size: metadata.size
    }
  }
}

// Tailles prédéfinies
const THUMBNAIL_SIZES = {
  small: { width: 150, height: 150 },
  medium: { width: 300, height: 300 },
  large: { width: 600, height: 600 }
}

Conversion Service ​

javascript
// services/conversion.service.js
const sharp = require('sharp')

class ConversionService {
  async convertToWebP(imageBuffer, quality = 85) {
    return await sharp(imageBuffer)
      .webp({ quality })
      .toBuffer()
  }

  async convertToJPEG(imageBuffer, quality = 90) {
    return await sharp(imageBuffer)
      .jpeg({ quality })
      .toBuffer()
  }

  async convertToPNG(imageBuffer) {
    return await sharp(imageBuffer)
      .png()
      .toBuffer()
  }

  async optimizeImage(imageBuffer, format) {
    const options = {
      jpeg: { quality: 85, progressive: true },
      png: { compressionLevel: 9 },
      webp: { quality: 85 }
    }

    return await sharp(imageBuffer)
      .toFormat(format, options[format])
      .toBuffer()
  }
}

Quota Service ​

javascript
// services/quota.service.js
class QuotaService {
  async getQuota(ecosystemId) {
    // Récupérer le quota depuis la base
    const quota = await prisma.storage.findUnique({
      where: { ecosystemId }
    })

    if (!quota) {
      throw new ApiError(404, 'Quota not found')
    }

    // Calculer l'utilisation réelle depuis GCP
    const files = await gcpService.listFiles(`ecosystems/${ecosystemId}`)
    const used = files.reduce((sum, file) => sum + file.size, 0)

    return {
      ecosystemId,
      maxSize: quota.maxSize,
      used: Math.round(used / (1024 * 1024)), // MB
      available: quota.maxSize - Math.round(used / (1024 * 1024)),
      percentage: (used / (quota.maxSize * 1024 * 1024)) * 100,
      files: files.length
    }
  }

  async checkQuota(ecosystemId, fileSize) {
    const quota = await this.getQuota(ecosystemId)
    const fileSizeMB = fileSize / (1024 * 1024)

    if (quota.available < fileSizeMB) {
      throw new ApiError(413, 'Storage quota exceeded')
    }

    return true
  }

  async updateUsage(ecosystemId, delta) {
    await prisma.storage.update({
      where: { ecosystemId },
      data: {
        used: {
          increment: delta
        }
      }
    })
  }
}

Middlewares ​

Upload Middleware (Multer) ​

javascript
// middlewares/upload.middleware.js
const multer = require('multer')

// Stockage en mémoire (buffer)
const storage = multer.memoryStorage()

const upload = multer({
  storage: storage,
  limits: {
    fileSize: 50 * 1024 * 1024 // 50 MB
  },
  fileFilter: (req, file, cb) => {
    const allowedTypes = [
      'image/jpeg',
      'image/png',
      'image/gif',
      'image/webp',
      'video/mp4',
      'video/webm',
      'application/pdf'
    ]

    if (allowedTypes.includes(file.mimetype)) {
      cb(null, true)
    } else {
      cb(new ApiError(400, 'Invalid file type'))
    }
  }
})

module.exports = {
  uploadSingle: upload.single('file'),
  uploadMultiple: upload.array('files', 10),
  uploadFields: upload.fields([
    { name: 'image', maxCount: 1 },
    { name: 'thumbnail', maxCount: 1 }
  ])
}

Quota Middleware ​

javascript
// middlewares/quota.middleware.js
const checkQuota = async (req, res, next) => {
  const { ecosystemId } = req.body
  const fileSize = req.file ? req.file.size : req.body.size

  try {
    await quotaService.checkQuota(ecosystemId, fileSize)
    next()
  } catch (error) {
    res.status(413).json({
      error: 'Storage quota exceeded',
      quota: await quotaService.getQuota(ecosystemId)
    })
  }
}

Événements AMQP ​

Événements Consommés ​

Routing KeyPayloadAction
roomee.storage.media{ ecosystemId, max_size, unit }Créer quota storage

Consumer :

javascript
// amqp/storageConsumer.js
amqpServer.subscribe('roomee.storage.media', async (message) => {
  const { ecosystemId, payload } = message

  await prisma.storage.create({
    data: {
      ecosystemId: ecosystemId,
      maxSize: payload.max_size,
      unit: payload.unit,
      used: 0
    }
  })

  logger.info('Storage quota created', { ecosystemId, maxSize: payload.max_size })
})

Configuration ​

Variables d'Environnement ​

bash
# Server
PORT=3003
NODE_ENV=development

# Database (minimal)
DATABASE_URL=mongodb://localhost:27017/roomee_media

# JWT
JWT_ACCESS_SECRET=your-jwt-secret

# Google Cloud Storage
GCP_PROJECT_ID=your-gcp-project-id
GCP_BUCKET_NAME=roomee-storage
GCP_KEY_FILE=./config/gcp-service-account-key.json

# Upload Limits
MAX_FILE_SIZE=52428800
MAX_CHUNK_SIZE=1048576

# AMQP
AMQP_GATEWAY_URL=amqp://localhost:5672
AMQP_EXCHANGE_NAME=roomee_events
AMQP_ROUTING_KEY_BASE=roomee.storage

Fichier GCP Service Account ​

json
{
  "type": "service_account",
  "project_id": "your-project",
  "private_key_id": "...",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
  "client_email": "storage@your-project.iam.gserviceaccount.com",
  "client_id": "...",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token"
}

Organisation GCP Bucket ​

roomee-storage/
├── ecosystems/
│   ├── {ecosystemId}/
│   │   ├── photos/
│   │   │   ├── photo1.jpg
│   │   │   └── thumbnails/
│   │   │       └── photo1-thumb.jpg
│   │   ├── documents/
│   │   │   └── menu.pdf
│   │   ├── videos/
│   │   │   └── presentation.mp4
│   │   └── avatars/
│   │       └── member-123.jpg
├── qr-codes/
│   └── hotel-{hotelId}.png
└── temp/
    └── chunks/
        └── upload-{uploadId}/

Tests ​

javascript
// __tests__/media.test.js
describe('MediaController', () => {
  describe('generatePresignedUrl', () => {
    it('should generate a presigned URL', async () => {
      const response = await request(app)
        .post('/media/generate-presigned-url')
        .set('Authorization', `Bearer ${token}`)
        .send({
          filename: 'test.jpg',
          contentType: 'image/jpeg',
          size: 1024000,
          ecosystemId: ecosystemId,
          folder: 'photos'
        })
        .expect(200)

      expect(response.body).toHaveProperty('uploadUrl')
      expect(response.body).toHaveProperty('fileUrl')
      expect(response.body).toHaveProperty('expiresAt')
    })
  })

  describe('uploadFile', () => {
    it('should upload a file to GCP', async () => {
      const response = await request(app)
        .post('/media/upload')
        .set('Authorization', `Bearer ${token}`)
        .attach('file', './test-files/image.jpg')
        .field('ecosystemId', ecosystemId)
        .field('folder', 'photos')
        .expect(200)

      expect(response.body).toHaveProperty('fileUrl')
      expect(response.body).toHaveProperty('thumbnailUrl')
    })
  })
})

Bonnes Pratiques ​

1. Toujours vérifier le quota avant upload ​

javascript
await quotaService.checkQuota(ecosystemId, fileSize)

2. Générer des thumbnails automatiquement pour les images ​

javascript
if (contentType.startsWith('image/')) {
  const thumbnail = await thumbnailService.generateThumbnail(fileBuffer)
  const thumbnailUrl = await gcpService.uploadFile(thumbnailPath, thumbnail)
}

3. Optimiser les images avant upload ​

javascript
const optimized = await conversionService.optimizeImage(fileBuffer, 'webp')

4. Nettoyer les fichiers temporaires ​

javascript
// Après upload par chunks
await cleanupTempFiles(uploadId)

5. Utiliser des presigned URLs pour les uploads ​

javascript
// Meilleur pour la performance
const { uploadUrl } = await generatePresignedUrl()
// Client upload directement vers GCP

Limitations et Améliorations Futures ​

Limitations Actuelles ​

  • ❌ Pas de CDN devant GCP Storage
  • ❌ Pas de compression vidéo
  • ❌ Pas de watermarking
  • ❌ Pas de détection de contenu inapproprié

Roadmap ​

  • [ ] Intégration CDN (Cloudflare/CloudFront)
  • [ ] Compression vidéo (FFmpeg)
  • [ ] Watermarking automatique
  • [ ] AI Content Moderation (Google Vision API)
  • [ ] Multi-région GCP
  • [ ] Support S3 (alternative à GCP)
  • [ ] Galleries et albums
  • [ ] Métadonnées EXIF preservation

Documentation technique Roomee Services