Functional Tests

Running PyDial Test Programs

  • run tests from the repository root by typing:

    >>> testPyDial
    

    This will run all tests within /tests/. The output should be similar to the following:

    PyDial> testPyDial
    Running PyDial Tests
      1 tests/test_DialogueServer.py   time 0m3.669s
      2 tests/test_Simulate.py         time 0m19.233s
      3 tests/test_Tasks.py            time 0m0.421s
    3 tests: 924 warnings,   0 errors
    See test logs in _testlogs for details
    

General Notes

  • These are not unit tests; they test higher capabilities, for example, they can initiate a simulate run. They are meant to be just basic tests to ensure that the repository is not broken. They should be run before git commits and pushes for example.

  • There are some odd looking python path adjustments at the start of the test_X.py files. These aren’t needed in general, but the tests don’t run on camdial for example without them. They look like this:

    >>> import os,sys
    >>> curdir = os.path.dirname(os.path.realpath(__file__))
    >>> curdir = curdir.split('/')
    >>> curdir = '/'.join(curdir[:-1]) +'/'
    >>> sys.path.append(curdir)
    

Other useful tools

  • The following python tools are really helpful for checking the speed and quality of your code:

  • pylint: detects outright errors but also gives suggestions on code styles and conventions. For example:

    >>> pylint Simulate.py
    
  • cProfile: Full profiling of every method called while executing some python code (including imported methods from the standard library etc). cProfile is part of the python std lib. For example:

    >>> python -m cProfile Simulate.py -c _config/simulate_multiDomains_HDC.cfg
    
  • profilehooks: use for profiling individual methods. Uses decorators to do so. Requires installing:

    >>> pip install profilehooks
    

List of Current Tests


test_DialogueServer.py - test DialogueServer()

Use utils/dummyDialogueServerClient to create multiprocess instances of a fake client which communicate concurrently with a running dialogue server in a separate process.

class tests.test_DialogueServer.TDialogueServer
test_dialogueserver()

Create a DialogueServer and a few dummy clients


test_Simulate.py - test simulation system

class tests.test_Simulate.TGPPolicy

GP (Gaussian Process) policy tests

test_simulate_multidomain()

Run 2 dialogues with the multidomain system and GP policies (NB: config needs to explicitly set each domain to use gp actually…)

test_simulate_singledomain_for_all_domains()

Loop over all domains and runs 2 dialogues with GP policy in each

class tests.test_Simulate.THDCPolicy

HDC (handcrafted) policy tests

test_simulate_multidomain()

Run HDC policy for domains in 2 multi domain dialogues

test_simulate_singledomain_for_all_domains()

Loop over all domains and run 2 dialogues via HDC policy in each


test_Tasks.py - test Task generator

class tests.test_Tasks.TTaskCreator
test_TasksCreator()

test Tasks.TaskCreator()

test_TasksReader()

test Tasks.TaskReader()