Conrad Fox

Programming django wagtail 

Wagtail: replicating a database with fixtures

Oct. 10, 2018

UPDATE: This post has recently been getting a lot of visits. I'm guessing recent changes to Wagtail have made using fixtures even harder. I have not had any luck replicating a database using dumpdata and loaddata with Wagtail 3.0+. I doesn't matter how many tables I exclude, whether I flush the database or whatever other incantation. Nothing works. If you're in the same boat, this post will not save you. For what it's worth, I now just use "pg_dump".

If you're just looking for something Wagtail-y to read, here's another post I wrote that still works.

Coarse Programmer's Guide to Faking Wagtail's Rich Text Entities

I just switched my blog over from Mezzanine toWagtail CMS. Mezzanine has served me well for several years. It does a lot, the design is logical and the documentation (and commenting) is beautifully articulate. I essentially learnt Python by wading through the Mezzanine code. But I detest the invisible knots the TinyMCE Rich Text editor ties you up in when you try to mix your <p>'s with any other tag. Wagtail offers StreamField, a substitute for the rich text blog, which lets you create compose content with custom HTML blocks within the editor. I was curious to try it out... so here I am.

The Wagtail Beginner's Miscellany isn't a comprehensive review of Wagtail for beginner's by any stretch. It is just a walk through of some of the hitches I encountered setting up a Wagtail site and how I solved them. Keep in mind this is written by a relative beginner for other beginners. There's nothing groundbreaking here, but they are the posts I wish existed a week ago.

Replicating your development database

When I develop a site, I usually start with real data in the database. The stories, photos, profiles, bios etc. already exist and it's easier to drop them in than try to invent a bunch of dummy entries. And by the time I deploy, I have entered so much data I don't want to do it again. I just want to replicate my development database in production using the dumpdata command, for example, by typing this on the development machine:

python dumpdata >> whatever1.json

And this on the production machine:

python loaddata whatever1.json

Unfortunately, tables created by Django and by Wagtail will contain initial data that clashes with the fixtures. You need to empty both sets of table on the production machine before you can import the fixture.

The "flush" management command should clear the Wagtail data, but, owing to this bug, a vestigial table with a ForeignKey to the Page model blocks it from working. When you try it, you get this error:

CommandError: Database sample_project couldn't be flushed. Possible reasons:
  * The database isn't running or isn't configured correctly.
  * At least one of the expected database tables doesn't exist.
  * The SQL was invalid.
Hint: Look at the output of 'django-admin sqlflush'. That's the SQL this command wasn't able to run.
The full error: cannot truncate a table referenced in a foreign key constraint
DETAIL:  Table "wagtailsearch_editorspick" references "wagtailcore_page".
HINT:  Truncate table "wagtailsearch_editorspick" at the same time, or use TRUNCATE ... CASCADE.

The table in question is related to an optional app called SearchPromotions which is no longer part of the the main Wagtail install. If you intend to use that app, then the following step may not work for you. Check out the bug report for more info. However, if you don't plan on using it -- I even know what it does -- just go ahead and get rid of the table with the following SQL command on the production database:

DROP TABLE wagtailsearch_editorspick;

Now you can flush the Wagtail data:

python flush

Note: you may need to add a settings option at the end of that line, for example "--settings=".

"Flush" does not eliminate data created in the Django tables, however. For that you will need this:

delete from auth_permission;
delete from django_content_type;

Finally, load the fixtures with "loaddata". Hope this helps someone!

I'm a journalist, producer, instructional designer and developer. I've reported from a minefield and a dugout canoe, taught radio journalism, robotics, and anthropology in board rooms, lecture halls and mud huts, led award-winning media projects, and mentored dozens of journalists in Canada and Latin America. I run a small media production company specialized in education and the environment called Studio Canek.

Digital storytelling

Working with Conrad brought out the best in me. He's very patient and very professional. He's really passionate about radio and he's always ready to venture into dangerous territory to get information that will make his story clear to his listeners.

Phares Jerome

Editor, Le Nouvelliste, Haiti