Skip to content

scheduling

scheduling

Admission to Ollama: interactive work precedes pending housekeeping.

This coordinates this process only. An inference already sent to Ollama is never pretended to be preemptible, nor are other programs' requests covered.

Classes

InferenceQueueTimeout

Bases: RuntimeError

The local admission queue timed out, not the Ollama connection.

BackgroundStopped

Bases: RuntimeError

The memory service stopped before its request reached Ollama.

InferenceScheduler 

InferenceScheduler(*, quiet_seconds: float = QUIET_SECONDS)
Source code in src/diapason/engine/scheduling.py 
87
88
89
90
91
92
93
94
def __init__(self, *, quiet_seconds: float = QUIET_SECONDS):
    self.quiet_seconds = quiet_seconds
    self._foreground = 0
    self._waiting = 0
    self._background_active = False
    self._pending: deque[_Ticket] = deque()
    self._last_foreground = 0.0
    self._model = ""

Functions:

interactive_turn 

interactive_turn()

Protect the whole chat, including gaps between tools and continuations.

Source code in src/diapason/engine/scheduling.py 
50
51
52
53
54
55
56
57
58
59
60
61
62
@contextmanager
def interactive_turn():
    """Protect the whole chat, including gaps between tools and continuations."""
    global _turns, _last_turn
    with _condition:
        _turns += 1
    try:
        yield
    finally:
        with _condition:
            _turns -= 1
            _last_turn = time.monotonic()
            _condition.notify_all()