parallel.controller.scheduler — IPython 0.11 documentation

### Navigation

- index
- modules |
- next |
- previous |
- home|
- search|
- documentation »
- The IPython API »

# parallel.controller.scheduler

## Module: `parallel.controller.scheduler`

Inheritance diagram for `IPython.parallel.controller.scheduler`:

The Python scheduler for rich scheduling.

The Pure ZMQ scheduler does not allow routing schemes other than LRU, nor does it check msg_id DAG dependencies. For those, a slightly slower Python Scheduler exists.

Authors:

- Min RK

## Class

## `TaskScheduler`

class `IPython.parallel.controller.scheduler.` `TaskScheduler`(**kwargs)

  Bases: `IPython.zmq.session.SessionFactory`

  Python TaskScheduler object.

  This is the simplest object that supports msg_id based DAG dependencies. Only task msg_ids are checked, not msg_ids of jobs submitted via the MUX queue.

  `__init__`(**kwargs)
  `add_job`(idx)

    Called after self.targets[idx] just got the job with header. Override with subclasses. The default ordering is simple LRU. The default loads are the number of outstanding jobs.

  `all_completed`

    An instance of a Python set.

  `all_done`

    An instance of a Python set.

  `all_failed`

    An instance of a Python set.

  `all_ids`

    An instance of a Python set.

  `audit_timeouts`()

    Audit all waiting tasks for expired timeouts.

  `auditor`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `blacklist`

    An instance of a Python dict.

  classmethod `class_config_section`()

    Get the config class config section

  classmethod `class_get_help`()

    Get the help string for this class in ReST format.

  classmethod `class_get_trait_help`(trait)

    Get the help string for a single trait.

  classmethod `class_print_help`()

    Get the help string for a single trait and print it.

  classmethod `class_trait_names`(**metadata)

    Get a list of all the names of this classes traits.

    This method is just like the `trait_names()` method, but is unbound.

  classmethod `class_traits`(**metadata)

    Get a list of all the traits of this class.

    This method is just like the `traits()` method, but is unbound.

    The TraitTypes returned don’t know anything about the values that the various HasTrait’s instances are holding.

    This follows the same algorithm as traits does and does not allow for any simple way of specifying merely that a metadata name exists, but has any value. This is because get_metadata returns None if a metadata key doesn’t exist.

  `client_stream`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `clients`

    An instance of a Python dict.

  `completed`

    An instance of a Python dict.

  `config`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `context`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `created` = None
  `depending`

    An instance of a Python dict.

  `destinations`

    An instance of a Python dict.

  `dispatch_notification`(msg)

    dispatch register/unregister events.

  `dispatch_result`(raw_msg)

    dispatch method for result replies

  `dispatch_submission`(raw_msg)

    Dispatch job submission to appropriate handlers.

  `engine_stream`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `fail_unreachable`(msg_id, why=<class 'IPython.parallel.error.ImpossibleDependency'>)

    a task has become unreachable, send a reply with an ImpossibleDependency error.

  `failed`

    An instance of a Python dict.

  `finish_job`(idx)

    Called after self.targets[idx] just finished a job. Override with subclasses.

  `graph`

    An instance of a Python dict.

  `handle_result`(idents, parent, raw_msg, success=True)

    handle a real task result, either success or failure

  `handle_stranded_tasks`(engine)

    Deal with jobs resident in an engine that died.

  `handle_unmet_dependency`(idents, parent)

    handle an unmet dependency

  `hwm`

    A integer trait.

  `ident`

    A casting version of the string trait.

  `loads`

    An instance of a Python list.

  `log`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `logname`

    A trait for unicode strings.

  `loop`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `maybe_run`(msg_id, raw_msg, targets, after, follow, timeout)

    check location dependencies, and run if they are met.

  `mon_stream`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `notifier_stream`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `on_trait_change`(handler, name=None, remove=False)

    Setup a handler to be called when a trait changes.

    This is used to setup dynamic notifications of trait changes.

    Static handlers can be created by creating methods on a HasTraits subclass with the naming convention ‘_[traitname]_changed’. Thus, to create static handler for the trait ‘a’, create the method _a_changed(self, name, old, new) (fewer arguments can be used, see below).

    Parameters : | handler : callable A callable that is called when a trait changes. Its signature can be handler(), handler(name), handler(name, new) or handler(name, old, new). name : list, str, None If None, the handler will apply to all traits. If a list of str, handler will apply to all names in the list. If a str, the handler will apply just to that name. remove : bool If False (the default), then install the handler. If True then unintall it.

  `pending`

    An instance of a Python dict.

  `resume_receiving`()

    Resume accepting jobs.

  `retries`

    An instance of a Python dict.

  `save_unmet`(msg_id, raw_msg, targets, after, follow, timeout)

    Save a message for later submission when its dependencies are met.

  `scheme`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `scheme_name`

    An enum that whose value must be in a given sequence.

  `session`

    A trait whose value must be an instance of a specified class.

    The value can also be an instance of a subclass of the specified class.

  `start`()
  `stop_receiving`()

    Stop accepting jobs while there are no engines. Leave them in the ZMQ queue.

  `submit_task`(msg_id, raw_msg, targets, follow, timeout, indices=None)

    Submit a task to any of a subset of our targets.

  `targets`

    An instance of a Python list.

  `trait_metadata`(traitname, key)

    Get metadata values for trait by key.

  `trait_names`(**metadata)

    Get a list of all the names of this classes traits.

  `traits`(**metadata)

    Get a list of all the traits of this class.

    The TraitTypes returned don’t know anything about the values that the various HasTrait’s instances are holding.

    This follows the same algorithm as traits does and does not allow for any simple way of specifying merely that a metadata name exists, but has any value. This is because get_metadata returns None if a metadata key doesn’t exist.

  `update_graph`(dep_id=None, success=True)

    dep_id just finished. Update our dependency graph and submit any jobs that just became runable.

    Called with dep_id=None to update entire graph for hwm, but without finishing a task.

## Functions

`IPython.parallel.controller.scheduler.` `launch_scheduler`(in_addr, out_addr, mon_addr, not_addr, config=None, logname='root', log_url=None, loglevel=10, identity='task', in_thread=False)

`IPython.parallel.controller.scheduler.` `leastload`(loads)

  Always choose the lowest load.

  If the lowest load occurs more than once, the first occurance will be used. If loads has LRU ordering, this means the LRU of those with the lowest load is chosen.

`IPython.parallel.controller.scheduler.` `logged`(f)

`IPython.parallel.controller.scheduler.` `lru`(loads)

  Always pick the front of the line.

  The content of loads is ignored.

  Assumes LRU ordering of loads, with oldest first.

`IPython.parallel.controller.scheduler.` `plainrandom`(loads)

  Plain random pick.

`IPython.parallel.controller.scheduler.` `twobin`(loads)

  Pick two at random, use the LRU of the two.

  The content of loads is ignored.

  Assumes LRU ordering of loads, with oldest first.

`IPython.parallel.controller.scheduler.` `weighted`(loads)

  Pick two at random using inverse load as weight.

  Return the less loaded of the two.

### Navigation

- index
- modules |
- next |
- previous |
- home|
- search|
- documentation »
- The IPython API »