Discourse Rake Posts Snippets from BBCode to Markdown

Snippets Jun 14, 2020

It's been more than 1 year since I migrated devilzc0de from MyBB to Discourse and many things needs to be done after the migration process is complete. One of them is to rake (and rebake, if needed) the BBCode tags to Markdown for each posts.

Before you start you need to:

  • Backup your forum before perform any actions because you may break every single posts content and PMs.
  • Prepare your time and resource, especially if your forum had lots of posts.
  • Know that Discourse use PostgreSQL  database which use POSIX regex to perform regex matching and it will be case-sensitive.
  • Know that you can use BBCode plugin for Discourse like discourse-bbcode without perform any of these actions.

If you understand the risks (and you have another option) and still want to continue without additional plugins like I do, then go ahead. You need to enter your Discourse app by executing ./launcher enter app from your root Discourse directory.

Remove Color, Size, Align, and Font Tags

Color, size, font and align tags may different in each post, like [color=#ff3333] or [color=red] , [font=‘Open Sans’, sans-serif] , [align=center] , etc. These rake post command will remove any general BBCode color, size, font, and align tags.

BBCode Color Tags

rake posts:delete_word['\\[color.*?\\]|\\[\\/color\\]',regex]

BBCode Size Tags

rake posts:delete_word['\\[size.*?\\]|\\[\\/size\\]',regex]

BBCode Align Tags

rake posts:delete_word['\\[align.*?\\]|\\[\\/align\\]',regex]

BBCode Font Tags

rake posts:delete_word['\\[font.*?\\]|\\[\\/font\\]',regex]

All of Them Together

rake posts:delete_word['\\[color.*?\\]|\\[\\/color\\]|\\[size.*?\\]|\\[\\/size\\]|\\[align.*?\\]|\\[\\/align\\]|\\[font.*?\\]|\\[\\/font\\]',regex]
Note: The regex is case-sensitive, tags with uppercase will not be removed. You need to check posts again by your self.

Replace String or Words (Remap)

You can use rake post:remap to perform simple replacement. For example, oldest devilzc0de forum post is from 2010 where at that time, we rarely found sites using https. But now, https is everywhere and browser won't display mixed content. So, to replace http:// to https:// for each posts :

rake posts:remap["http://","https://"]

This works for tagging user tags and emoji too.

Note: You may increase sidekiq workers by updating Discourse app.yml  UNICORN_SIDEKIQS env to spin up sidekiq queue for onebox and remote image fetch after remapping http to https.

Unfortunately, rake remap doesn't work for [code], [spoiler], [php], and [qoute] tags. Here is the example :

In old MyBB forum, user using [code] block following with the actual codes without new line will work. But, markdown code tags works if you use new line after ``` block.

[code]var promise = new Promise(function(resolve, reject) {
  // do a thing, possibly async, then…

  if (/* everything is fine */) {
    resolve("Worked!");
  }
  else {
    reject(Error("Didn't work"));
  }
});
[/code]

[code] tags above will work on MyBB and produce:

var promise = new Promise(function(resolve, reject) {
  // do a thing, possibly async, then…

  if (/* everything is fine */) {
    resolve("Worked!");
  }
  else {
    reject(Error("Didn't work"));
  }
});

But on Discourse which use Markdown, if I replace [code] tags with ``` will produce :

```var promise = new Promise(function(resolve, reject) {
  // do a thing, possibly async, then…

  if (/* everything is fine */) {
    resolve("Worked!");
  }
  else {
    reject(Error("Didn't work"));
  }
});
```

Code above will not recognised as code block. We need to add new line after first opening ``` block. Unfortunately (at least for me), using rake posts:remap["[code]","```\n"] doesn't work and only produce the "\n" as plain text. The only working solution for me is directly update raw PG database.

PostgreSQL Raw Update

You need to enter PostgreSQL on your Discourse app to run all PG query below. If you don't know how to do it, run su -c 'psql discourse' postgres  after ./launcher enter app command.

Note: You need to rebake your post after update PostgreSQL post DB. To avoid rebake all your posts, you can use rebake_match. See example below.

Replace Spoiler Tags With Details Tags

UPDATE posts set raw=replace(raw, '[spoiler]', '[details=Spoiler]'||chr(10)) where raw like '%[spoiler]%';
UPDATE posts set raw=replace(raw, '[spoiler=', '[details=') where raw like '%[spoiler=%';
UPDATE posts set raw=replace(raw, '[/spoiler]', chr(10)||'[/details]') where raw like '%[/spoiler]%';

After updating database, exit from postgres user and run rebake post.

rake posts:rebake_match["\[details"]

Replace MyBB [php] Tags With Markdown PHP Syntax Highlight

UPDATE posts set raw=replace(raw, '[php]', '```php'||chr(10)) where raw like '%[php]%';
UPDATE posts set raw=replace(raw, '[/php]', chr(10)||'```') where raw like '%[/php]%';

After updating database, exit from postgres user and run rebake post.

rake posts:rebake_match["\`\`\`php"]

Replace [code] tags With Markdown Tags

UPDATE posts set raw=replace(raw, '[code]', '```'||chr(10)) where raw like '%[code]%';
UPDATE posts set raw=replace(raw, '[/code]', chr(10)||'```') where raw like '%[/code]%';

After updating database, exit from postgres user and run rebake post.

rake posts:rebake_match["\`\`\`"]

To Do's

  • [ul], [ol] and [li] tags replace to Markdown list tags
  • [quote] Tags to Markdown quote tags

Please share if you have done mass replace for those two to do lists above.

Credits:

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.