Multisite configuration with Docker

You may wish to host multiple domains on a singled Docker setup. To do so follow these instructions:

We strongly recommend you run separate web and data containers

The standalone container is extremely easy to configure, however has a couple of drawbacks

1. You will need to stop your site when bootstrapping a new image
2. You run through the db bootstrap only when needed

Multisite is a fairly advanced topic, learn about hooks before attempting this.

Understand hooks

Discourse templates use pups, its rules are simple and powerful.

Each rule you run may define a hook:

run:
  exec:
    cd: some/path
    hook: my_hook
    cmd:
      - echo 1

Later on in your container you can insert rules before or after a hook:

hooks:
  before_my_hook:
    - exec: echo "I ran before"
  after_my_hook:
     - exec: echo "I ran after"

So in the example above you will see output like the following:

I ran before
1
I ran after

You can read through the templates in /var/discourse/templates to see what hooks you have available.

Amend your standalone container to provision the site and talk to it

Replace the entire hooks section with:

hooks:
  after_postgres:
     - exec: sudo -u postgres createdb b_discourse || exit 0
     - exec:
          stdin: |
            grant all privileges on database b_discourse to discourse;
          cmd: sudo -u postgres psql b_discourse
          raise_on_fail: false

     - exec: /bin/bash -c 'sudo -u postgres psql b_discourse <<< "alter schema public owner to discourse;"'
     - exec: /bin/bash -c 'sudo -u postgres psql b_discourse <<< "create extension if not exists hstore;"'
     - exec: /bin/bash -c 'sudo -u postgres psql b_discourse <<< "create extension if not exists pg_trgm;"'

  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - mkdir -p plugins
          - git clone https://github.com/discourse/docker_manager.git
  before_bundle_exec:
    - file:
        path: $home/config/multisite.yml
        contents: |
         secondsite:
           adapter: postgresql
           database: b_discourse
           pool: 25
           timeout: 5000
           db_id: 2
           host_names:
             - b.discourse.example.com

  after_bundle_exec:
    - exec: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

There are 3 hooks in play:

1. after_postgres, ensures that after postgres is installed an additional db called b_discourse is created with the appropriate permissions.

2. before_bundle_exec, ensures docker_manager is in place and that the multisite.yml file is in place (which defines where to find the databases)

3. after_bundle_exec, runs the custom db migration tast rake multisite:migrate this ensures all the dbs are up to date.

The above sample can be split into data container / app container if needed (just run the after_postgres hook in the data container and the rest in web container)

The above sample can be extended to provision evne more DBs, to do so, provision more dbs by duplicating the create db etc calls, and make sure you have additional sites in multisite.yml

Make sure you amend the host_names node in multisite.yml to match the actual host name you wish to host.

So I tried to follow these instructions yesterday, but I quickly got lost. I wonder if we could make this tutorial a little bit more informative for newbies.

I successfully installed discourse using docker on Digital Ocean at (let's call it) class1.example.com. (I followed these instructions: https://github.com/discourse/discourse/blob/master/docs/INSTALL-digital-ocean.md)

Then, because, I was hoping to be able to have a second site on the same Digital Ocean droplet called class2.example.com, I followed the link at the bottom of that tutorial to this page.

The first thing I don't understand is the initial "bold" recommendation.

It's not clear to me whether, by installing discourse with Docker (as per the referenced instructions), I am running separate web and data containers or not. The rest of the instructions seem to assume that I have separate containers. So, how can I tell. If I don't have separate containers, then I need some instruction about how to do this.

If I navigate to /var/docker/containers, am I in my data container?
If I am in /var/docker, and then run ./launcher ssh app, and then navigate to var/www/discourse am In my web container?

As you can see the difference between the web and data container is unclear to me. Again, having already successfully installed discourse with docker, have I already created a separate data container?

This is unclear to me. Obviously, not knowing where my "data container" is, this is difficult for me to do. Secondly, this looks like a .yml syntax, not just a shell script, so what exactly do you mean by 'run.'? Should I create a new file called multisite.yml or should I include these commands in my var/docker/containers/app.yml and then re-run ./launcher bootstrap app.yml?

This doesn't seem hard except I don't know where my web container is.

My questions may reveal a lot of ignorance. But like I said, I knew enough to do the initial install of a single site with docker. It feels like there is a big jump in assumed knowledge between those instructions and these.

I would really appreciate any help you can give me.

That is absolutely right, heh. Start out by reading this:

@riking Thanks. This is a helpful start.

I've set up a multisite using separate data and web containers. For this I added the mentioned sections to data.yml and web_only.yml. I needed to add host: <ip>, username: discourse and password: <pass> to the secondsite section:

  after_code:
    - file:
        path: /var/www/discourse/config/multisite.yml
        contents: |
         secondsite:
           adapter: postgresql
           database: b_discourse
           username: discourse
           password: 123456789yeahright
           pool: 25
           timeout: 5000
           host: 123.45.67.89
           db_id: 2
           host_names:
             - b.discourse

It seems to be working now.

Is this the best way of running more than one discourse site? How does multisite affect upgrading? Can I still use the docker plugin? How does this affect backups? Will I be able to move to separate containers/servers in the future?

EDIT: would this be a better way to setup two discourse instances on the same host - using two standalone containers and an nginx proxy?

The official Discourse hosted service actually uses both multisite and separate containers. So both are fine.

That's slightly trickier - IIRC the docker manager plugin makes admins of both sites able to run an upgrade that affects both.

Already fixed - backup filenames contain the site name.

I don't think it's mentioned here, but in going from standalone.yml to multisite, you need to make sure to keep redis, cron and syslog templates in your app.yml file. I made the mistake of removing those and modifying the file to make it exactly like the web_only.yml.

It would really help to have multisite.yml in the samples directory of discourse_docker.

Sam,
One substantive question regarding the howto and two small recommendations. First, the substantive question. You mentioned that we "should start by running a dedicated data container..." But I googled this and didn't come up with any articles on this topic. Can you elaborate on exactly what a dedicated data container is and how such is configured?
Now the small recommendations:
- You have "This will ensure a db called b_discourse i provisioned correctly." - "i" should be "is"
- You have "down in the web container and run the migrations code." - "run" should be "runs"

Ok so I tried to follow this here, but somehow got lost. I have discourse setup as outlined in the (docker installation guide). There we have to configure the containers/app.yml file. Where do I add the commands mentioned above? In the app.yml file or copy over data and web config files?

PS: I manage many servers and know Linux quite well, just new to the docker/discourse world. Thanks.

The commands above are in app.yml

Have no success with this at all. I edit the app.yml. Added suggestions like username and password to it, but still always fails with the following messages:

Tasks: TOP => db:migrate => environment
(See full trace by running task with --trace)
I, [2014-08-17T20:49:29.569593 #45]  INFO -- :
2014-08-17 20:49:29 UTC LOG:  received smart shutdown request
[176 | signal handler] (1408308569) Received SIGTERM, scheduling shutdown...
2014-08-17 20:49:29 UTC LOG:  autovacuum launcher shutting down
2014-08-17 20:49:29 UTC LOG:  shutting down
[176] 17 Aug 20:49:29.577 # User requested shutdown...
[176] 17 Aug 20:49:29.577 * Saving the final RDB snapshot before exiting.
2014-08-17 20:49:29 UTC LOG:  database system is shut down
[176] 17 Aug 20:49:29.729 * DB saved on disk
[176] 17 Aug 20:49:29.730 # Redis is now ready to exit, bye bye...
/pups/lib/pups/exec_command.rb:85:in `spawn': cd /var/www/discourse && sudo -E -u discourse bundle exec rake db:migrate failed with return #<Process::Status: pid 388 exit 1> (RuntimeError)
    from /pups/lib/pups/exec_command.rb:55:in `block in run'
    from /pups/lib/pups/exec_command.rb:53:in `each'
    from /pups/lib/pups/exec_command.rb:53:in `run'
    from /pups/lib/pups/command.rb:6:in `run'
    from /pups/lib/pups/config.rb:85:in `block (2 levels) in run_commands'
    from /pups/lib/pups/config.rb:76:in `each'
    from /pups/lib/pups/config.rb:76:in `block in run_commands'
    from /pups/lib/pups/config.rb:75:in `each'
    from /pups/lib/pups/config.rb:75:in `run_commands'
    from /pups/lib/pups/config.rb:71:in `run'
    from /pups/lib/pups/cli.rb:31:in `run'
    from /pups/bin/pups:8:in `<main>'
02bbf45b4e1f7d8250930affc21c52664a1b412fe5e13ff24d33a1e1f557034a
FAILED TO BOOTSTRAP

The following is now in my app.yml:

hooks:
  after_postgres:
    - exec:
        stdin: |
          alter user discourse with password 'secret';
        cmd: sudo -u postgres psql discourse
        raise_on_fail: false

hooks:
   after_postgres:
     - exec: sudo -u postgres createdb b_discourse || exit 0
     - exec:
          stdin: |
            grant all privileges on database b_discourse to discourse;
          cmd: sudo -u postgres psql b_discourse
          raise_on_fail: false

     - exec: /bin/bash -c 'sudo -u postgres psql b_discourse <<< "alter schema public owner to discourse;"'
     - exec: /bin/bash -c 'sudo -u postgres psql b_discourse <<< "create extension if not exists hstore;"'
     - exec: /bin/bash -c 'sudo -u postgres psql b_discourse <<< "create extension if not exists pg_trgm;"'

#multisite
hooks:
  after_code:
    - file:
        path: /var/www/discourse/config/multisite.yml
        contents: |
         secondsite:
           adapter: postgresql
           database: b_discourse
           username: discourse
           password: secret
           pool: 25
           timeout: 5000
           db_id: 2
           host_names:
             - 2.domain.com

    - exec:
        cd: /var/www/discourse/plugins
        cmd:
          - mkdir -p plugins
          - git clone https://github.com/discourse/docker_manager.git
  after_web:
    - exec: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

Any help is greatly appreciated.

As a follow-up. I've done all the git pull and updates to docker already. Still no success.

I tried a multisite setup on my test server and I don't get it. Did all the reading linked in this topic.
I understand that to run separated data and web containers I copy samples/data.yml and web_only.yml to containers/web.yml, edit with my configuration and .launcher bootstrap data, ./launcher bootstrap web then ./launcher start data and ./launcher start web.

I get only errors:

/var/discourse# ./launcher start web
...
Unable to find image 'local_discourse/web' locally
Pulling repository local_discourse/web
2014/10/17 11:59:27 Error: image local_discourse/web not found

/var/discourse# ./launcher start data
...
(<unknown>): did not find expected '-' indicator while parsing a block collection at line 36 column 5 -e LANG=en_US.UTF-8
YAML syntax error. Please check your /var/docker/containers/*.yml config files.

The YAML syntax error is strange because line 36 col 5 of data.yml is exactly a "-". There are no tabs or excess spaces as far I can see. The first error suggests that I am doin it wrong.

What did @dirkcuys do exactly?

I really like Discourse since the migration of my first community has been a success. I have enough memory and cpu to spin a second forum instance for a separate community on the same server. If I don't understand how to do multisite, I can still use NGINX in front of multiple discourse instances, but I'd really like to be able to evaluate both alternatives.

Hey @blau - I didn't do anything more than said, but I did do it a while ago (before Discourse 1.x) and havent updated yet. The mentioned sections I added are those mentioned by @sam in the first post.

If I get around to updating that server before this question gets answered and I encounter anything different, I'll let you know.

Hi @dirkcuys, did you copy samples/data.yml and web_only.yml to containers/ or did you somehow create an containers/app.yml including those?

You are trying to start a container "web" but your container will be "web_only" unless you rename the yml file.

Thank you @brahn , in fact I had renamed web_only.yml to web.yml. I have updated my message for coherence.

Yes, that is exactly what I did!

I was experimenting with this solution (nginx proxy before two Discourse instances) and discovered at my own espense that it fails painfully. Don't do it!

What happened:
Configure nginx as a reverse proxy, firewall the related ports, test that it works.
Two instances on the same host: /var/discourse and /var/discourse2.
Start the first instance, /var/discourse
Start the second instance /var/discourse2: fail!

It seems that /var/discourse is hard referenced in the Discourse Docker repository, for example do a grep -r "/var/discourse/" /var/discourse2 on the host.

So when I started the app in the second instance /var/discourse2 it messed plentily in the first instance /var/discourse. Fortunately I was able to recover a rsnapshot backup of the first instance and restored it from there.

So, unless the canonical path is replaced by a user-configurable variable, I have to make the original multisite work.

I think you should rather set up two Docker images, iow copy /var/discourse/samples/standalone.yml to /var/discourse/containers/app1.yml and /var/discourse/containers/app2.yml.