[Product-Developers] Mock testing

5 Messages

Martin Aspeli-2

Reply Threaded

Hi guys,

I just wanted to share with you that I *love* mocker and mock based
testing. :-)

I blogged about it here:

http://martinaspeli.net/articles/mock-testing-with-mocker-and-plone.mocktestcase

I just released plone.mocktestcase 1.0b1:

http://pypi.python.org/pypi/plone.mocktestcase

I'd like get some input on the above article and opinions from anyone
who tries this. I'd like to merge the article above into
plone.org/documentation/tutorial/testing at some point too.

Cheers,
Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

_______________________________________________
Product-Developers mailing list
Product-Developers@...
http://lists.plone.org/mailman/listinfo/product-developers

Maurits van Rees-3

[Product-Developers] Re: Mock testing

Reply Threaded

Martin Aspeli, on 2008-06-29:

> Hi guys,
>
> I just wanted to share with you that I *love* mocker and mock based
> testing. :-)
>
> I blogged about it here:
>
> http://martinaspeli.net/articles/mock-testing-with-mocker-and-plone.mocktestcase
>
> I just released plone.mocktestcase 1.0b1:
>
> http://pypi.python.org/pypi/plone.mocktestcase
>
> I'd like get some input on the above article and opinions from anyone
> who tries this. I'd like to merge the article above into
> plone.org/documentation/tutorial/testing at some point too.

Hi Martin,

I have not tried any code (also not from Dexterity), only read your
article. Some things are not clear to me.

- At the end of the first mock example you say: "We must remember to put
the test case into replay mode, using self.replay()." Why? What does
that do?

- In the third mock example you say that we "set the expectation that
lookup_schema() should be called on it once", stressing the word
'once'. Which part of the test code does that? From the paragraphs
after that I *guess* that whenever the code is this:

self.expect(...).result(...)

it implicitly means this:

self.expect(...).result(...).count(1, 1)

Is that a correct guess? If so, I think you should mention that
explicitly, perhaps already in the first mock example.

BTW, if that is correct then I would find "count(0, None)" a more
logical default, so by default no restriction on the number of times
a mocked method is called.

And a question: is there next to mock_utility also something like
mock_tool, so you could you this to mock for example the
portal_catalog? I guess something along these lines might work:

from whereever import CatalogTool
catalog_mock = self.mocker.proxy(CatalogTool())
context_mock = self.mocker.mock()
self.expect(context_mock.portal_catalog).result(catalog_mock).count(0,None)

--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Product-Developers mailing list
Product-Developers@...
http://lists.plone.org/mailman/listinfo/product-developers

Martin Aspeli-2

[Product-Developers] Re: Mock testing

Reply Threaded

Hi Maurits,

> I have not tried any code (also not from Dexterity), only read your
> article. Some things are not clear to me.
>
> - At the end of the first mock example you say: "We must remember to put
> the test case into replay mode, using self.replay()." Why? What does
> that do?

Before you put the mock into replay mode, it's in "record" mode. All
operations you do on it are essentially expectations. For example:

mymock = self.mocker.mock()
self.expect(mymock.foo()).result(True)

Now, if I do

mymock.foo()

without putthing the mock in replay mode, it's basically just recording
an expectation that foo will be called. In fact, the above could have
been rewritten as:

mymock = self.mocker.mock()
mymock.foo()
self.mocker.result(True)

though I prefer the self.expect() chaining syntax as it's more compact.

Only when the mock is put in replay mode will it actually exhibit the
behaviour and check that it's being called in the expected way. You do
that with

self.replay()

This is how most mock libraries work. You start off in record mode,
doing operations on the object that "record" expectations about how it
should be called. You then put it into "replay" mode and call the code
under test (initialised to use the mock, obviously). The mock library
then checks that recorded operations actually happen (in the way that
they were recorded) and throws assertion errors if they're not. Finally,
you do a verify + restore (implicit in the unit test) that ensures no
steps were missed and unpatches anything patched for the test.

> - In the third mock example you say that we "set the expectation that
> lookup_schema() should be called on it once", stressing the word
> 'once'. Which part of the test code does that? From the paragraphs
> after that I *guess* that whenever the code is this:
>
> self.expect(...).result(...)
>
> it implicitly means this:
>
> self.expect(...).result(...).count(1, 1)

>
> Is that a correct guess?

Yes.

> If so, I think you should mention that
> explicitly, perhaps already in the first mock example.

In record mode, the assumption is that if you record an action once, you
meant for it to happen once. Let's say I have a mock and I expect it to
be called twice, returning one thing the first time and something else
the second time:

magic_eight_ball = self.mocker.mock()
self.expect(magic_eight_ball()).result("No")
self.expect(magic_eight_ball()).result("Yes")

> BTW, if that is correct then I would find "count(0, None)" a more
> logical default, so by default no restriction on the number of times
> a mocked method is called.

Most mock libraries don't work like this. They fail when a method is
called too many times (and will tell you so). I think it's (marginally)
better to be explicit if you don't care how many times something's
called. It's likely that if you expected something to be called, and
it's called 0 times, then it's an error, and if it's called dozens of
times, it's also an error.

> And a question: is there next to mock_utility also something like
> mock_tool, so you could you this to mock for example the
> portal_catalog? I guess something along these lines might work:
>
> from whereever import CatalogTool
> catalog_mock = self.mocker.proxy(CatalogTool())
> context_mock = self.mocker.mock()
> self.expect(context_mock.portal_catalog).result(catalog_mock).count(0,None)

No. There could be, but bear in mind that mock tests are not running in
a PloneTestCase (which is why they're so quick) - at least not normally
(no reason they couldn't be, but then you're really doing an integration
test). As such, if your code calls getToolByName(), you'd probably do a
replace() on this.

A generic mock_tool() could look like this, though:

from mocker import ANY

class MockTestCase(...):
...
getToolByName_mock = None

def mock_tool(self, tool_mock, name, expected_context=ANY,
min=1, max=None):
mock = self.getToolByName_mock
if mock is None:
self.getToolByName_mock = mock = \
self.mocker.replace('Products.CMFCore.utils.getToolByName')
self.expect(mock(expected_context,
name)).result(tool_mock).count(min, max)

Untested and possibly refactorable of course.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

_______________________________________________
Product-Developers mailing list
Product-Developers@...
http://lists.plone.org/mailman/listinfo/product-developers

David Bain-5

Re: [Product-Developers] Re: Mock testing

Reply Threaded

Wow

* easy to understand
* quick to write
* fast to run

Contrary to what I may have indicated in the past, I have actually written products with tests, while the tests were valuable, they were generally:

* challenging to read
* hard to write
* slow to run

As a result, I have tended to write fewer tests than I should on projects.

If this thing works as advertised, I may begin to "sup" at the testing "table" more often.

On Wed, Jul 2, 2008 at 3:41 PM, Martin Aspeli <optilude@...> wrote:

Hi Maurits,

I have not tried any code (also not from Dexterity), only read your
article. Some things are not clear to me.

- At the end of the first mock example you say: "We must remember to put
the test case into replay mode, using self.replay()." Why? What does
that do?

Before you put the mock into replay mode, it's in "record" mode. All operations you do on it are essentially expectations. For example:

mymock = self.mocker.mock()
self.expect(mymock.foo()).result(True)

Now, if I do

mymock.foo()

without putthing the mock in replay mode, it's basically just recording an expectation that foo will be called. In fact, the above could have been rewritten as:

mymock = self.mocker.mock()
mymock.foo()
self.mocker.result(True)

though I prefer the self.expect() chaining syntax as it's more compact.

Only when the mock is put in replay mode will it actually exhibit the behaviour and check that it's being called in the expected way. You do that with

self.replay()

This is how most mock libraries work. You start off in record mode, doing operations on the object that "record" expectations about how it should be called. You then put it into "replay" mode and call the code under test (initialised to use the mock, obviously). The mock library then checks that recorded operations actually happen (in the way that they were recorded) and throws assertion errors if they're not. Finally, you do a verify + restore (implicit in the unit test) that ensures no steps were missed and unpatches anything patched for the test.

- In the third mock example you say that we "set the expectation that
lookup_schema() should be called on it once", stressing the word
'once'. Which part of the test code does that? From the paragraphs
after that I *guess* that whenever the code is this:

self.expect(...).result(...)

it implicitly means this:

self.expect(...).result(...).count(1, 1)

>

Is that a correct guess?

Yes.

If so, I think you should mention that
explicitly, perhaps already in the first mock example.

In record mode, the assumption is that if you record an action once, you meant for it to happen once. Let's say I have a mock and I expect it to be called twice, returning one thing the first time and something else the second time:

magic_eight_ball = self.mocker.mock()
self.expect(magic_eight_ball()).result("No")
self.expect(magic_eight_ball()).result("Yes")

BTW, if that is correct then I would find "count(0, None)" a more
logical default, so by default no restriction on the number of times
a mocked method is called.

Most mock libraries don't work like this. They fail when a method is called too many times (and will tell you so). I think it's (marginally) better to be explicit if you don't care how many times something's called. It's likely that if you expected something to be called, and it's called 0 times, then it's an error, and if it's called dozens of times, it's also an error.

And a question: is there next to mock_utility also something like
mock_tool, so you could you this to mock for example the
portal_catalog? I guess something along these lines might work:

from whereever import CatalogTool
catalog_mock = self.mocker.proxy(CatalogTool())
context_mock = self.mocker.mock()
self.expect(context_mock.portal_catalog).result(catalog_mock).count(0,None)

No. There could be, but bear in mind that mock tests are not running in a PloneTestCase (which is why they're so quick) - at least not normally (no reason they couldn't be, but then you're really doing an integration test). As such, if your code calls getToolByName(), you'd probably do a replace() on this.

A generic mock_tool() could look like this, though:

from mocker import ANY

class MockTestCase(...):
...
getToolByName_mock = None

def mock_tool(self, tool_mock, name, expected_context=ANY,
min=1, max=None):
mock = self.getToolByName_mock
if mock is None:
self.getToolByName_mock = mock = \
self.mocker.replace('Products.CMFCore.utils.getToolByName')
self.expect(mock(expected_context,
name)).result(tool_mock).count(min, max)

Untested and possibly refactorable of course.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

_______________________________________________
Product-Developers mailing list
Product-Developers@...
http://lists.plone.org/mailman/listinfo/product-developers

_______________________________________________
Product-Developers mailing list
Product-Developers@...
http://lists.plone.org/mailman/listinfo/product-developers

Martin Aspeli-2

[Product-Developers] Re: Mock testing

Reply Threaded

Hi David,

> * easy to understand
> * quick to write
> * fast to run
>
> Contrary to what I may have indicated in the past, I have actually
> written products with tests, while the tests were valuable, they were
> generally:
>
> * challenging to read
> * hard to write
> * slow to run
>
> As a result, I have tended to write fewer tests than I should on projects.
>
> If this thing works as advertised, I may begin to "sup" at the testing
> "table" more often.

Heh, that can only be a good thing. :)

That said, mock testing is no silver bullet. I find that it is very
helpful to systematically test different inputs and edge conditions. It
also helps understand how your code works and what dependencies it has,
because you need to understand that quite well to write effective mocks.

It is not, however, a replacement for all integration testing. It's
quite possible to write mock tests that pass even if the code is wrong,
and there's a danger that you end up just writing your code twice - once
in mock syntax and once for the implementation - so that in the end
you're not really testing anything. As with all testing, mock testing
requires some skill and experience.

On another (non-Python) project, we use mock based testing for all
service components and logic, unit tests running with database
clean-down for DAO integration testing, and Selenium tests for
end-to-end functional integration testing. Our Selenium tests tend to
cover only the major code paths, but ensure that we haven't missed some
wiring or forgotten a major piece of functionality. Mock-based unit
tests then ensure that each component works to its specification.

I find this approach quite effective, and it's something we could
consider for Plone. Selenium tests are slow to run and fairly fragile
(because they rely on the rendered state of the page), but easy to
record. Incidentally, I wrote collective.ploneseltest
(http://pypi.python.org/pypi/collective.ploneseltest) a while back to
support Selenium tests in Plone that are based on PloneTestCase setup
and the Python Selenium remote control API (as opposed to the somewhat
difficult-to-automate HTML table syntax).

For Zope, of course, we also have zope.testbrowser, although it can be
hard to write testbrowser tests when you need to abstractly think about
how the rendered Plone page is going to work. The test recorder
obviously helps.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

_______________________________________________
Product-Developers mailing list
Product-Developers@...
http://lists.plone.org/mailman/listinfo/product-developers