Development environment with Mongo cluster


Transactions in MongoDB are allowed only if using cluster. This means that even on local development environment a Mongo cluster should be started and used. This project is a Hapi.js service that uses a local MongoDB cluster. Service can be started in many ways:
  • Start service using an external Mongo replicaset. Just need to provide correct DB URI in .env (or as environment variable).
  • Start service using the MongoDB replicaset started with docker-compose.
  • Start service and MongoDB using docker-compose.
  • Start service and MongoDB using minikube (soon).
Service
Start using docker-compose

Service

Here is the github repository for the demo service. This service exposes an API for some basic operations. One of the main reasons behind creating this service was to have a startershell HapiJs project including best practices that I am usually adding to all my projects. All those main features are presented bellow.
Configuration
  • Configuration is loaded from environment variables.
  • Support for loading from an `.env` file is provided. However the `dotenv` is added as dev dependency making sure that `.env` is not loaded in production as this will be a bad practice.
  • Before service is started a strict schema validation is applied on configuration object. This is required to make sure the application is started with a valid configuration, at least from structure point of view.
  • There are no configuration defaults. This will enforce creating proper environment variables for all configuration parameters.

.env
    SERVICE_PORT=3000
    SERVICE_HOST=0.0.0.0

    PROJECT_NAME=Startershell
    NODE_ENV=development
    DB=mongodb://mongo0:27017,mongo1:27017,mongo2:27017/startershell?replicaSet=rs0
                                    

Swagger documentation
  • Swagger documentation for implemented API is exposed automatically on `/documentation` endpoint.
  • This documentation is created automatically using the HapiJs endpoint validation so is up-to-date and automatically updated when API is changed.
  • This documentation is exposed automatically only for `development` environments, in `staging` or `production` it will no exposed API documentation.
  • Project can be started also using minikube (soon).
Endpoint validation
  • All endpoints have a strict validation implementing using JOI.
  • This implementation will be automatically described in the Swagger documentation.
  • Validation errors details are not exposed in the HTTP response. The validation error reason is only exposed in log.
Logging
  • Logging is done using 'hapi-pino', one of the fastest logger available for HapiJs.
  • Logger uses a pretty log style only in development environments.
Database
  • MongoDB is used for this project. Because I wanted to use transactions I needed to use a Mongo replicaset. The docker-compose provided in this project is creating the required MongoDB replicaset.

Start using docker-compose

Docker
Docker files are provided for:
  • development environment - Dockerfile.dev. This is using nodemon for automatically reloading the service when changes are made.
  • production environment - Dockerfile
Configuration files are presented bellow:
Dockerfile
    FROM mongo:4.1.13-bionic
    ADD ./bin/replicate.js /replicate.js
    ADD ./bin/setup.sh /setup.sh
    CMD ["sh", "setup.sh"]
                                    

setup.sh
    #!/usr/bin/env sh

    if [ -f /replicated.txt ]; then
        echo "Mongo is already set up"
    else
        echo "Setting up mongo replication and seeding initial data..."
        sleep 10s
        mongo mongo0:27017 replicate.js
        echo "Replication done..."
        # Wait for few seconds until replication takes effect
        touch /replicated.txt
    fi
                                    

replicate.js
                                    rs.initiate( {
                                        _id : "rs0",
                                        members: [
                                            { _id: 0, host: "mongo0:27017", priority: 1 },
                                            { _id: 1, host: "mongo1:27017", priority: 0.5 },
                                            { _id: 2, host: "mongo2:27017", priority: 0.5 },
                                        ]
                                    });
                                    
By using these files the MongoDB replicaset can be properly configured and started using docker-compose.
Docker Compose
A docker compose is provided. This docker compose contains our service and also a MongoDB replicaset. The docker-compose will:
  • start multiple Mongo nodes
  • configure them as a replicaset
  • start service
docker-compose.yml
    version: "3"
    services:
      service:
        build:
          context: .
          dockerfile: Dockerfile.dev
        image: service:local
        container_name: service
        env_file:
          - env-variables.env
        ports:
          - 3000:3000
        depends_on:
          - mongo-conf
        volumes:
          - .:/app

      mongo0:
        hostname: mongo0
        container_name: mongo0
        image: mongo
        ports:
          - 27017:27017
        restart: always
        entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
      mongo1:
        hostname: mongo1
        container_name: mongo1
        image: mongo
        ports:
          - 27018:27017
        restart: always
        entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
      mongo2:
        hostname: mongo2
        container_name: mongo2
        image: mongo
        ports:
          - 27019:27017
        restart: always
        entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
      mongo-conf:
        container_name: mongo-conf
        build:
          context: .
          dockerfile: Dockerfile-mongo-conf
        depends_on:
          - mongo0
          - mongo1
          - mongo2
                                    

If you want to start mongo cluster but not the service (so you can then start service in debug mode locally) you can use this command:
sh
    docker-compose up --build --scale service=0