Skip to content

lablink_client_service.monitoring.samplers.filesystem

Filesystem sampler.

Polls the watch dir recursively for the newest .slp (HDF5) and the newest training_log.csv under any models/ directory. Recursion matters: start.sh clones the tutorial repository into a subfolder of the watch dir, and SLEAP writes models/ next to the project file — a top-level-only scan misses everyone who follows the tutorial layout. Returns numeric progress signals only — no filenames, no paths, no file contents leave this module.

A partially-written .slp may raise on open; we swallow and return None for that field, letting the next tick try again.

Source code in packages/client/src/lablink_client_service/monitoring/samplers/filesystem.py 
48
49
50
51
52
53
54
55
56
57
def count_labeled_frames(slp: Path) -> int | None:
    try:
        with h5py.File(slp, "r") as f:
            ds = f.get(FRAMES_DATASET)
            if ds is None:
                return None
            return int(ds.shape[0])
    except (OSError, KeyError, ValueError) as e:
        logger.debug("h5py read of %s failed: %s", slp, e)
        return None
Source code in packages/client/src/lablink_client_service/monitoring/samplers/filesystem.py 
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
def parse_training_log(log: Path) -> tuple[int | None, float | None]:
    try:
        with log.open() as f:
            rows = list(csv.DictReader(f))
    except OSError as e:
        logger.debug("training_log read failed: %s", e)
        return None, None
    if not rows:
        return None, None
    last = rows[-1]
    try:
        epoch = int(float(last.get("epoch", "0")))
    except (TypeError, ValueError):
        return None, None
    return epoch, _pick_loss(last)
Source code in packages/client/src/lablink_client_service/monitoring/samplers/filesystem.py 
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def sample(watch_dir: str) -> tuple[int | None, int | None, float | None]:
    root = Path(watch_dir)
    if not root.exists():
        return None, None, None
    # ponytail: full directory walk every tick; cache/inotify if a large
    # dataset under the watch dir ever makes this visible in profiles.
    slp = _latest(root.glob("**/*.slp"))
    frames = count_labeled_frames(slp) if slp is not None else None

    log = _latest(root.glob("**/models/**/training_log.csv"))
    epoch, loss = (None, None)
    if log is not None:
        epoch, loss = parse_training_log(log)

    return frames, epoch, loss