Backend for the Slurm workload manager. More...

#include <SlurmBackend.h>

Inheritance diagram for benchmax::SlurmBackend:

Collaboration diagram for benchmax::SlurmBackend:

[legend]

Public Member Functions
bool	suspendable () const

void	run (const Jobs &jobs, bool wait_for_termination)
	Run all tools on all benchmarks using Slurm. More...

void	process_results (const Jobs &jobs, bool check_finished)

void	addResult (const Tool *tool, const fs::path &file, BenchmarkResult &&result)
	Add a result. More...

Protected Member Functions
virtual void	startTool (const Tool *)
	Hook for every tool at the beginning. More...

virtual void	finalize ()
	Hook to allow for asynchronous backends to wait for jobs to terminate. More...

virtual void	execute (const Tool *, const fs::path &)
	Execute a single pair of tool and benchmark. More...

void	madeProgress (std::size_t files=1)
	Can be called to give information about the current progress, if available. More...

void	sanitize_results (const Jobs &jobs) const

void	write_results (const Jobs &jobs) const

Protected Attributes
std::size_t	mExpectedJobs
	Number of jobs that should be run. More...

std::atomic< std::size_t >	mFinishedJobs
	Number of jobs that are finished. More...

std::atomic< std::size_t >	mLastPercent
	Percentage of finished jobs when madeProgress() was last called. More...

Private Types
using	JobData = std::tuple< const Tool *, std::filesystem::path, BenchmarkResult >
	A job consists of a tool, an input file, a base dir and results. More...

Private Member Functions
void	parse_result_file (const Jobs &jobs, const std::filesystem::path &file, std::map< size_t, JobData > &results)
	Parse the content of an output file. More...

std::pair< std::size_t, std::size_t >	get_job_range (std::size_t n, std::size_t numJobs) const

void	store_job_id (int jobid)

std::vector< int >	load_job_ids ()

void	remove_job_ids ()

void	run_job_async (std::size_t n, const std::vector< JobData > &results, bool wait_for_termination)

bool	collect_results (const Jobs &jobs, bool check_finished) override

Private Attributes
std::mutex	mSubmissionMutex
	Mutex for submission delay. More...

std::mutex	mSlurmjobMutex
	Mutex for slurmjobs file. More...

Results	mResults
	Results of already completed jobs. More...

Detailed Description

Backend for the Slurm workload manager.

The execution model is as follows: We create multiple jobs that each consists of multiple array jobs that each execute one slice of the task list. One array job executes Settings::slice_size entries of the task list. One job consists of Settings::array_size array jobs. We start as many jobs as necessary.

Definition at line 29 of file SlurmBackend.h.

Member Typedef Documentation

◆ JobData

using benchmax::SlurmBackend::JobData = std::tuple< const Tool*, std::filesystem::path, BenchmarkResult >

private

A job consists of a tool, an input file, a base dir and results.

Definition at line 32 of file SlurmBackend.h.

Member Function Documentation

◆ addResult()

void benchmax::Backend::addResult	(	const Tool *	tool,
		const fs::path &	file,
		BenchmarkResult &&	result
	)

inlineinherited

Add a result.

Definition at line 97 of file Backend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ collect_results()

bool benchmax::SlurmBackend::collect_results	(	const Jobs &	jobs,
		bool	check_finished
	)

inlineoverrideprivatevirtual

Reimplemented from benchmax::Backend.

Definition at line 177 of file SlurmBackend.h.

Here is the call graph for this function:

◆ execute()

virtual void benchmax::Backend::execute	(	const Tool *	,
		const fs::path &
	)

inlineprotectedvirtualinherited

Execute a single pair of tool and benchmark.

Reimplemented in benchmax::LocalBackend, and benchmax::CondorBackend.

Definition at line 49 of file Backend.h.

Here is the caller graph for this function:

◆ finalize()

virtual void benchmax::Backend::finalize ( )

inlineprotectedvirtualinherited

Hook to allow for asynchronous backends to wait for jobs to terminate.

Definition at line 45 of file Backend.h.

Here is the caller graph for this function:

◆ get_job_range()

std::pair<std::size_t,std::size_t> benchmax::SlurmBackend::get_job_range	(	std::size_t	n,
		std::size_t	numJobs
	)		const

inlineprivate

Definition at line 97 of file SlurmBackend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ load_job_ids()

std::vector<int> benchmax::SlurmBackend::load_job_ids ( )

inlineprivate

Definition at line 113 of file SlurmBackend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ madeProgress()

void benchmax::Backend::madeProgress ( std::size_t files = 1 )

inlineprotectedinherited

Can be called to give information about the current progress, if available.

Definition at line 51 of file Backend.h.

Here is the caller graph for this function:

◆ parse_result_file()

void benchmax::SlurmBackend::parse_result_file	(	const Jobs &	jobs,
		const std::filesystem::path &	file,
		std::map< size_t, JobData > &	results
	)

inlineprivate

Parse the content of an output file.

Definition at line 44 of file SlurmBackend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ process_results()

void benchmax::Backend::process_results	(	const Jobs &	jobs,
		bool	check_finished
	)

inlineinherited

Definition at line 88 of file Backend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ remove_job_ids()

void benchmax::SlurmBackend::remove_job_ids ( )

inlineprivate

Definition at line 127 of file SlurmBackend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ run()

void benchmax::SlurmBackend::run	(	const Jobs &	jobs,
		bool	wait_for_termination
	)

inline

Run all tools on all benchmarks using Slurm.

Definition at line 222 of file SlurmBackend.h.

Here is the call graph for this function:

◆ run_job_async()

void benchmax::SlurmBackend::run_job_async	(	std::size_t	n,
		const std::vector< JobData > &	results,
		bool	wait_for_termination
	)

inlineprivate

Definition at line 133 of file SlurmBackend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ sanitize_results()

void benchmax::Backend::sanitize_results ( const Jobs & jobs ) const

inlineprotectedinherited

Definition at line 61 of file Backend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ startTool()

virtual void benchmax::Backend::startTool ( const Tool * )

inlineprotectedvirtualinherited

Hook for every tool at the beginning.

Can be used to upload the tool to some remote system.

Definition at line 41 of file Backend.h.

Here is the caller graph for this function:

◆ store_job_id()

void benchmax::SlurmBackend::store_job_id ( int jobid )

inlineprivate

Definition at line 105 of file SlurmBackend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ suspendable()

bool benchmax::SlurmBackend::suspendable ( ) const

inline

Definition at line 218 of file SlurmBackend.h.

◆ write_results()

void benchmax::Backend::write_results ( const Jobs & jobs ) const

inlineprotectedinherited

Definition at line 70 of file Backend.h.

Here is the call graph for this function:

Here is the caller graph for this function:

Field Documentation

◆ mExpectedJobs

std::size_t benchmax::Backend::mExpectedJobs

protectedinherited

Number of jobs that should be run.

Definition at line 29 of file Backend.h.

◆ mFinishedJobs

std::atomic<std::size_t> benchmax::Backend::mFinishedJobs

protectedinherited

Number of jobs that are finished.

Definition at line 31 of file Backend.h.

◆ mLastPercent

std::atomic<std::size_t> benchmax::Backend::mLastPercent

protectedinherited

Percentage of finished jobs when madeProgress() was last called.

Definition at line 33 of file Backend.h.

◆ mResults

Results benchmax::Backend::mResults

privateinherited

Results of already completed jobs.

Definition at line 26 of file Backend.h.

◆ mSlurmjobMutex

std::mutex benchmax::SlurmBackend::mSlurmjobMutex

private

Mutex for slurmjobs file.

Definition at line 41 of file SlurmBackend.h.

◆ mSubmissionMutex

std::mutex benchmax::SlurmBackend::mSubmissionMutex

private

Mutex for submission delay.

Definition at line 39 of file SlurmBackend.h.

The documentation for this class was generated from the following file:

benchmax/backends/SlurmBackend.h

Public Member Functions

Protected Member Functions

Protected Attributes

Private Types

Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

◆ JobData

Member Function Documentation

◆ addResult()

◆ collect_results()

◆ execute()

◆ finalize()

◆ get_job_range()

◆ load_job_ids()

◆ madeProgress()

◆ parse_result_file()

◆ process_results()

◆ remove_job_ids()

◆ run()

◆ run_job_async()

◆ sanitize_results()

◆ startTool()

◆ store_job_id()

◆ suspendable()

◆ write_results()

Field Documentation

◆ mExpectedJobs

◆ mFinishedJobs

◆ mLastPercent

◆ mResults

◆ mSlurmjobMutex

◆ mSubmissionMutex