Faster rebuilds?

Hey guys, I wonder if there is a way to run ./launcher rebuild app a bit faster. Right now it takes 6-9 minutes to do the whole stuff and i'd like to skip some stuff that looks a bit useless:

These three commands takes a lot of time (about 20-30-60 sec each)

• cd /var/www/discourse && chown -R discourse /var/www/discourse
• cd /var/www/discourse && su discourse -c 'bundle install --deployment --verbose --without test --without development'
• cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'

Compiling assets for each locale is also take about a minute, uglifying js is also takes some time.

Aaaaanyhooow, I was hoping there is a way of semi-rebuilding (?!) the app a bit faster than that. I understand the need of running all of these when you're installing an update or a new plugin, but sometimes I only need a very quick „restart” for more... special cases:

• update of a SSL cert
• the other day i tried to make it play nice with other sites (following this article) and i had to rebuild couple of times.

PS: time ./launcher rebuild app --skip-prereqs is giving only negligible improvements:

real    5m27.461s
user    0m1.027s
sys     0m0.652s

Thanks!

If rebuild time is a huge concern to you I would strongly recommend running a dedicated data container

That way you can bootstrap while your existing site is up and running, meaning you only deal with a few seconds of outage when upgrading via launcher

Bundle install is usually very fast when we just ship an image, I plan on having more regular updates to help out here

Assets precompile is the slowest part, we are cosidering pre-bundling assets or possibly adding a caching server or even more aggressive image updates, but nothing is a trivially easy solution

Is there any migration documentation about how to change to this configuration? I thought I saw something at one point but can't find it in #howto now.

This one. Noted some steps is out-dated:

1. Use enter to get into a container.
2. DISCOURSE_DB_HOST: data and DISCOURSE_REDIS_HOST: data
   • link to a container is prefered than IP now.
   • define links block as follows:

links:
  - link:
      name: data
      alias: data

if we don't stop forum during rebuild (make it read-only if necessary), it will be better?

Can not be done in a single container setup without major headaches... part of the bootstrap process is spawning the db, you can't have two instances of a database running at the same time.

How does this work when using a separate data container? Does it run migrations on the data that the old site is still using, potentially breaking the site until the web container is upgraded if the migrations are incompatible?

Yes, but..... We are very careful to keep migrations compatible data wise, even have a pretty elaborate scheme that delays dropping columns when we move them to new tables

Should this process have copied my data & configuration over? I ran through the steps but got a clean/"fresh" site, so rolled back to the single-container version.

I did a restore from a backup made just before the process, and it seemed to restore everything. So probably this just needs to be made clear. :slightly_smiling:

I flagged it to ask for wikifing the topic... Let's amend it

PS: Don't know whether it's a right way to use flag..

Yes, that's the right way to use the notify moderator feature

Can you say a bit more about how this process works in your mind?

After switching to the multi-container scenario, I just did a ./launcher rebuild web_only but my downtime was the same. Clearly I need more coffee still this morning.

./launcher bootstrap web_only

Start outage

./launcher destroy web_only
./launcher start web_only

End outage

Is there any way to skip the whole uglify and compressing of assets & locales during bootstrap if the assets already exist?

Maybe it would make sense to have a dedicated assets volume and/or a comparison of checksums of input files and output filenames before a recompile is attempted.

Or utilising the cache of a docker image build on bootstrap, which could cache the whole git pull / install process unless git repo changes ( https://stackoverflow.com/a/39278224 )

Could checksums be used here @sam or @falco?

Yes, I discussed this with @eviltrout in the past, we would need a post build step, an asset server and to customize asset precompile

I estimate it would take less than 2 days to setup

In the event of a failure in the start method, how can you roll back to the previous container?