ONNXPredictorBase Class Reference

Owns the ONNX Runtime session of one model file, for the predictors built on it. More...

#include <OpenMS/ML/ONNX/ONNXPredictorBase.h>

Public Member Functions
	ONNXPredictorBase (const std::string &model_path, int threads=4)
	Constructor initializes the generic ONNX session safely across platforms.
virtual	~ONNXPredictorBase ()
	ONNXPredictorBase (const ONNXPredictorBase &)=delete
ONNXPredictorBase &	operator= (const ONNXPredictorBase &)=delete
	ONNXPredictorBase (ONNXPredictorBase &&) noexcept
ONNXPredictorBase &	operator= (ONNXPredictorBase &&) noexcept
Ort::Session &	session ()
	Access the wrapped ONNX Runtime session.
const Ort::Session &	session () const
Ort::MemoryInfo &	memoryInfo ()
	Access CPU memory info used to construct input tensors.
const Ort::MemoryInfo &	memoryInfo () const
std::vector< std::string >	getInputNames () const
	Return model input names in ONNX graph order.
std::vector< std::string >	getOutputNames () const
	Return model output names in ONNX graph order.
std::vector< int64_t >	getInputShape (size_t input_index) const
	Return the declared input shape for an input index.
size_t	getInputCount () const
	Return the number of model inputs.
size_t	getOutputCount () const
	Return the number of model outputs.
void	forEachBatch (const std::vector< std::vector< size_t > > &groups, size_t batch_size, const std::function< void(const size_t *indices, size_t n)> &run) const
	Runs run on each batch of groups, up to the constructor's threads batches at once (with OpenMP)

Private Attributes
int	threads_ = 1
std::unique_ptr< Ort::SessionOptions >	session_options_
std::unique_ptr< Ort::Session >	session_
std::unique_ptr< Ort::MemoryInfo >	memory_info_

Detailed Description

Owns the ONNX Runtime session of one model file, for the predictors built on it.

Loads the model with all graph optimizations, without ONNX Runtime's memory pattern (the inputs of the models change shape from call to call), and gives a predictor (PeptDeepRTInference, PeptDeepCCSInference, PeptDeepMS2Inference) the session and the CPU memory info it needs to build input tensors. The ONNX Runtime types are only forward-declared here. Available in builds with WITH_ONNX.

A predictor splits its input into model runs (calls of Ort::Session::Run(), one per batch of peptides of one length) and runs them with forEachBatch(): up to the constructor's threads runs at once, each on one thread (the session has one intra-op thread). On CPU, the small models of PeptDeep scale much better over their calls than within one call (2.2x faster with 16 threads), and the result of a call does not depend on the number of its threads.

Constructor & Destructor Documentation

ONNXPredictorBase() [1/3]

ONNXPredictorBase ( const std::string &  model_path, int  threads = 4  )

explicit

Constructor initializes the generic ONNX session safely across platforms.

Parameters

model_path	Path to the ONNX model file.
threads	The model runs forEachBatch() runs at once (each on one thread); at least 1.

~ONNXPredictorBase()

virtual ~ONNXPredictorBase ( )

virtual

ONNXPredictorBase() [2/3]

ONNXPredictorBase ( const ONNXPredictorBase &  )

delete

ONNXPredictorBase() [3/3]

ONNXPredictorBase ( ONNXPredictorBase &&  )

noexcept

Member Function Documentation

forEachBatch()

void forEachBatch	(	const std::vector< std::vector< size_t > > &	groups,
		size_t	batch_size,
		const std::function< void(const size_t *indices, size_t n)> &	run
	)		const

Runs run on each batch of groups, up to the constructor's threads batches at once (with OpenMP)

The batches of a group are its consecutive slices of at most batch_size indices (the last one shorter); run(indices, n) gets one of them, e.g. to run the session on those peptides (Ort::Session::Run() may be called concurrently) and store their predictions. Once all have run, the first exception of a batch, if any, is rethrown.

getInputCount()

size_t getInputCount

(

)

const

Return the number of model inputs.

getInputNames()

std::vector< std::string > getInputNames

(

)

const

Return model input names in ONNX graph order.

getInputShape()

std::vector< int64_t > getInputShape

(

size_t

input_index

)

const

Return the declared input shape for an input index.

getOutputCount()

size_t getOutputCount

(

)

const

Return the number of model outputs.

getOutputNames()

std::vector< std::string > getOutputNames

(

)

const

Return model output names in ONNX graph order.

memoryInfo() [1/2]

Ort::MemoryInfo & memoryInfo

(

)

Access CPU memory info used to construct input tensors.

memoryInfo() [2/2]

const Ort::MemoryInfo & memoryInfo

(

)

const

operator=() [1/2]

ONNXPredictorBase & operator= ( const ONNXPredictorBase &  )

delete

operator=() [2/2]

ONNXPredictorBase & operator= ( ONNXPredictorBase &&  )

noexcept

session() [1/2]

Ort::Session & session

(

)

Access the wrapped ONNX Runtime session.

session() [2/2]

const Ort::Session & session

(

)

const

Member Data Documentation

memory_info_

std::unique_ptr<Ort::MemoryInfo> memory_info_

private

session_

std::unique_ptr<Ort::Session> session_

private

session_options_

std::unique_ptr<Ort::SessionOptions> session_options_

private

threads_

int threads_ = 1

private