Skip to content

panoptes.pipeline.status

panoptes.pipeline.status

The stages a frame and an observation move through.

These describe this pipeline's work -- CALIBRATING, SOLVING, MATCHING, EXTRACTING -- and they used to live in panoptes-data, the client that reads what this pipeline produces. That is the wrong way round: the reader held the writer's vocabulary, so the writer could not be changed without changing the reader, and the writer did not import them at all once the Firestore path was deleted. They belong here, versioned with the data contract. See data contract 5.2.

Both are IntEnum because the comparison is the point: the idempotency rule is "at or past PROCESSING", not a set membership test. The order below is therefore load-bearing, and the members before PROCESSING are the ones that mean "not done" -- including ERROR, so a failed frame compares as unprocessed and gets picked up again.

ImageStatus

Bases: IntEnum

The stage one frame has reached.

Anything below PROCESSING means the frame still needs work.

Source code in src/panoptes/pipeline/status.py 
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
class ImageStatus(IntEnum):
    """The stage one frame has reached.

    Anything below `PROCESSING` means the frame still needs work.
    """

    ERROR = auto()
    MASKED = auto()
    UNKNOWN = auto()
    RECEIVING = auto()
    RECEIVED = auto()
    UNSOLVED = auto()
    PROCESSING = auto()
    CALIBRATING = auto()
    CALIBRATED = auto()
    SOLVING = auto()
    SOLVED = auto()
    MATCHING = auto()
    MATCHED = auto()
    EXTRACTING = auto()
    EXTRACTED = auto()

ObservationStatus

Bases: IntEnum

The stage one observation has reached.

Source code in src/panoptes/pipeline/status.py 
46
47
48
49
50
51
52
53
54
55
56
57
58
59
class ObservationStatus(IntEnum):
    """The stage one observation has reached."""

    ERROR = auto()
    NOT_ENOUGH_FRAMES = auto()
    UNKNOWN = auto()
    CREATED = auto()
    RECEIVING = auto()
    RECEIVED = auto()
    PROCESSING = auto()
    CALIBRATING = auto()
    CALIBRATED = auto()
    MATCHING = auto()
    MATCHED = auto()

image_status 

image_status(name: object) -> ImageStatus

Read a status name from a document, tolerating whatever is actually there.

A document written by an older pipeline, or one that never recorded a status, reads as ImageStatus.UNKNOWN rather than raising -- UNKNOWN sorts below PROCESSING, so an unreadable status means the frame is reprocessed. Failing closed here would strand every frame written before statuses were recorded.

The argument is typed object on purpose: it comes from JSON that this pipeline may not have written, so it can be any type at all. A list is unhashable and would make ImageStatus[name] raise TypeError rather than KeyError, which is how one malformed document could stop a walk over the whole archive.

Source code in src/panoptes/pipeline/status.py 
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def image_status(name: object) -> ImageStatus:
    """Read a status name from a document, tolerating whatever is actually there.

    A document written by an older pipeline, or one that never recorded a
    status, reads as `ImageStatus.UNKNOWN` rather than raising -- `UNKNOWN`
    sorts below `PROCESSING`, so an unreadable status means the frame is
    reprocessed. Failing closed here would strand every frame written before
    statuses were recorded.

    The argument is typed `object` on purpose: it comes from JSON that this
    pipeline may not have written, so it can be any type at all. A list is
    unhashable and would make ``ImageStatus[name]`` raise `TypeError` rather
    than `KeyError`, which is how one malformed document could stop a walk over
    the whole archive.
    """
    if not isinstance(name, str):
        return ImageStatus.UNKNOWN
    try:
        return ImageStatus[name]
    except KeyError:
        return ImageStatus.UNKNOWN