Module Cuda.Graph

CUDA graphs: capture a sequence of stream operations once, then replay it with a single launch call. See: Graph Management.

type t

A graph under construction (a template of work, not yet executable). The graph value retains the OCaml-side resources of the operations captured into it (kernels — and through them their modules — kernel parameters, device pointers), so a Stream.synchronize of the capture stream cannot free resources a replay still needs. See CUgraph.

type exec

An instantiated, executable graph; retains the captured operations' resources like t. See CUgraphExec.

val sexp_of_t : t -> Sexplib0.Sexp.t
val sexp_of_exec : exec -> Sexplib0.Sexp.t
type capture_mode =
  1. | GLOBAL
  2. | THREAD_LOCAL
  3. | RELAXED
    (*

    See CUstreamCaptureMode. GLOBAL (the CUDA default) invalidates the capture on potentially unsafe API calls from any thread; THREAD_LOCAL restricts that to the capturing thread; RELAXED imposes no cross-thread restrictions.

    *)
val sexp_of_capture_mode : capture_mode -> Sexplib0.Sexp.t
val begin_capture : ?mode:capture_mode -> Stream.t -> unit

Starts capturing work submitted to the stream instead of executing it. mode defaults to GLOBAL. Between begin_capture and end_capture, operations enqueued on the stream (e.g. Stream.launch_kernel) are recorded as graph nodes, not run. See cuStreamBeginCapture.

val end_capture : Stream.t -> t

Ends the capture begun by begin_capture and returns the captured graph. Raises Cuda_error if the capture was invalidated. The graph value is finalized using cuGraphDestroy; it can also be released eagerly with destroy. See cuStreamEndCapture.

val instantiate : t -> exec

Creates an executable graph from a captured graph; the template can be destroyed afterwards. The exec value is finalized using cuGraphExecDestroy; it can also be released eagerly with exec_destroy, but only once no launch of it is pending (synchronize the stream first). See cuGraphInstantiate.

val launch : exec -> Stream.t -> unit

Enqueues the whole captured work sequence on the stream, as one API call. The stream retains the exec until it is synchronized (mirroring Stream.launch_kernel's argument retention), so dropping the last reference to a launched exec cannot destroy it while the launch is pending. See cuGraphLaunch.

val destroy : t -> unit

Eagerly destroys the graph template. Idempotent; the finalizer covers the non-eager case.

val exec_destroy : exec -> unit

Eagerly destroys the executable graph. Idempotent; the finalizer covers the non-eager case.