6: Storing Resources In ZODB¶
Store and retrieve resource tree containers and items in a database.
Background¶
We now have a resource tree that can go infinitely deep, adding items and subcontainers along the way. We obviously need a database, one that can support hierarchies. ZODB is a transaction-based Python database that supports transparent persistence. We will modify our application to work with the ZODB.
Along the way we will add the use of pyramid_tm
, a system for adding
transaction awareness to our code. With this we don't need to manually manage
our transaction begin/commit cycles in our application code. Instead,
transactions are setup transparently on request/response boundaries, outside
our application code.
Objectives¶
- Create a CRUD app that adds records to persistent storage.
- Setup
pyramid_tm
andpyramid_zodbconn
. - Make our "content" classes inherit from
Persistent
. - Set up a database connection string in our application.
- Set up a root factory that serves the root from ZODB rather than from memory.
Steps¶
We are going to use the previous step as our starting point:
$ cd ..; cp -r addcontent zodb; cd zodb
Introduce some new dependencies in
zodb/setup.py
:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
from setuptools import setup requires = [ 'pyramid', 'pyramid_jinja2', 'ZODB3', 'pyramid_zodbconn', 'pyramid_tm', 'pyramid_debugtoolbar' ] setup(name='tutorial', install_requires=requires, entry_points="""\ [paste.app_factory] main = tutorial:main """, )
We can now install our project:
$ $VENV/bin/python setup.py develop
Modify our
zodb/development.ini
to include some configuration and give database connection parameters:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
[app:main] use = egg:tutorial pyramid.reload_templates = true pyramid.includes = pyramid_debugtoolbar pyramid_zodbconn pyramid_tm zodbconn.uri = file://%(here)s/Data.fs?connection_cache_size=20000 [server:main] use = egg:pyramid#wsgiref host = 0.0.0.0 port = 6543 # Begin logging configuration [loggers] keys = root, tutorial [logger_tutorial] level = DEBUG handlers = qualname = tutorial [handlers] keys = console [formatters] keys = generic [logger_root] level = INFO handlers = console [handler_console] class = StreamHandler args = (sys.stderr,) level = NOTSET formatter = generic [formatter_generic] format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s # End logging configuration
Our startup code in
zodb/tutorial/__init__.py
gets some bootstrapping changes:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
from pyramid.config import Configurator from pyramid_zodbconn import get_connection from .resources import bootstrap def root_factory(request): conn = get_connection(request) return bootstrap(conn.root()) def main(global_config, **settings): config = Configurator(settings=settings, root_factory=root_factory) config.include('pyramid_jinja2') config.scan('.views') return config.make_wsgi_app()
Our views in
zodb/tutorial/views.py
have modest changes inadd_folder
andadd_content
for how new instances are made and put into a container:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
from random import randint from pyramid.httpexceptions import HTTPFound from pyramid.location import lineage from pyramid.view import view_config from .resources import ( Root, Folder, Document ) class TutorialViews(object): def __init__(self, context, request): self.context = context self.request = request self.parents = reversed(list(lineage(context))) @view_config(renderer='templates/root.jinja2', context=Root) def root(self): page_title = 'Quick Tutorial: Root' return dict(page_title=page_title) @view_config(renderer='templates/folder.jinja2', context=Folder) def folder(self): page_title = 'Quick Tutorial: Folder' return dict(page_title=page_title) @view_config(name='add_folder', context=Folder) def add_folder(self): # Make a new Folder title = self.request.POST['folder_title'] name = str(randint(0, 999999)) new_folder = Folder(title) new_folder.__name__ = name new_folder.__parent__ = self.context self.context[name] = new_folder # Redirect to the new folder url = self.request.resource_url(new_folder) return HTTPFound(location=url) @view_config(name='add_document', context=Folder) def add_document(self): # Make a new Document title = self.request.POST['document_title'] name = str(randint(0, 999999)) new_document = Document(title) new_document.__name__ = name new_document.__parent__ = self.context self.context[name] = new_document # Redirect to the new document url = self.request.resource_url(new_document) return HTTPFound(location=url) @view_config(renderer='templates/document.jinja2', context=Document) def document(self): page_title = 'Quick Tutorial: Document' return dict(page_title=page_title)
Make our resources persistent in
zodb/tutorial/resources.py
:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
from persistent import Persistent from persistent.mapping import PersistentMapping import transaction class Folder(PersistentMapping): def __init__(self, title): PersistentMapping.__init__(self) self.title = title class Root(Folder): __name__ = None __parent__ = None class Document(Persistent): def __init__(self, title): Persistent.__init__(self) self.title = title def bootstrap(zodb_root): if not 'tutorial' in zodb_root: root = Root('My Site') zodb_root['tutorial'] = root transaction.commit() return zodb_root['tutorial']
No changes to any templates!
Run your Pyramid application with:
$ $VENV/bin/pserve development.ini --reload
Open http://localhost:6543/ in your browser.
Analysis¶
We install pyramid_zodbconn
to handle database connections to ZODB. This
pulls the ZODB3 package as well.
To enable pyramid_zodbconn
:
- We activate the package configuration using
pyramid.includes
. - We define a
zodbconn.uri
setting with the path to the Data.fs file.
In the root factory, instead of using our old root object, we now get a connection to the ZODB and create the object using that.
Our resources need a couple of small changes. Folders now inherit from
persistent.PersistentMapping
and document from persistent.Persistent
.
Note that Folder
now needs to call super()
on the __init__
method,
or the mapping will not initialize properly.
On the bootstrap, note the use of transaction.commit()
to commit the
change. This is because on first startup, we want a root resource in place
before continuing.
ZODB has many modes of deployment. For example, ZEO is a pure-Python object storage service across multiple processes and hosts. RelStorage lets you use a RDBMS for storage/retrieval of your Python pickles.
Extra Credit¶
- Create a view that deletes a document.
- Remove the configuration line that includes
pyramid_tm
. What happens when you restart the application? Are your changes persisted across restarts? - What happens if you delete the files named
Data.fs*
?