Package: BatchJobs 1.9

Bernd Bischl

BatchJobs: Batch Computing with R

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF, SLURM and Sun Grid Engine. Multicore and SSH systems are also supported. For further details see the project web page.

Authors:Bernd Bischl <[email protected]>, Michel Lang <[email protected]>, Henrik Bengtsson <[email protected]>

BatchJobs_1.9.tar.gz
BatchJobs_1.9.zip(r-4.5)BatchJobs_1.9.zip(r-4.4)BatchJobs_1.9.zip(r-4.3)
BatchJobs_1.9.tgz(r-4.4-any)BatchJobs_1.9.tgz(r-4.3-any)
BatchJobs_1.9.tar.gz(r-4.5-noble)BatchJobs_1.9.tar.gz(r-4.4-noble)
BatchJobs_1.9.tgz(r-4.4-emscripten)BatchJobs_1.9.tgz(r-4.3-emscripten)
BatchJobs.pdf |BatchJobs.html✨
BatchJobs/json (API)
NEWS

# Install 'BatchJobs' in R:
install.packages('BatchJobs', repos = c('https://tudo-r.r-universe.dev', 'https://cloud.r-project.org'))

Peer review:

Bug tracker:https://github.com/tudo-r/batchjobs/issues

On CRAN:

105 exports 85 stars 4.22 score 25 dependencies 3 dependents 2 mentions 616 scripts 1.2k downloads

Last updated 2 years agofrom:14f00881e5. Checks:OK: 4 ERROR: 3. Indexed: yes.

TargetResultDate
Doc / VignettesOKSep 13 2024
R-4.5-winERRORSep 13 2024
R-4.5-linuxOKSep 13 2024
R-4.4-winERRORSep 13 2024
R-4.4-macOKSep 13 2024
R-4.3-winERRORSep 13 2024
R-4.3-macOKSep 13 2024

Exports:addRegistryPackagesaddRegistrySourceDirsaddRegistrySourceFilesapplyJobFunctionbatchExpandGridbatchExportbatchMapbatchMapQuickbatchMapResultsbatchQuerybatchReducebatchReduceResultsbatchUnexportcallFunctionOnSSHWorkerscfBrewTemplatecfHandleUnknownSubmitErrorcfKillBatchJobcfReadBrewTemplatecheckIdscopyRequiredJobFilesdbCreateJobDefTabledbGetJobsdebugMulticoredebugSSHfilterResultsfindDisappearedfindDonefindErrorsfindExpiredfindJobsfindMissingResultsfindNotDonefindNotErrorsfindNotOnSystemfindNotRunningfindNotStartedfindNotSubmittedfindNotTerminatedfindOnSystemfindRunningfindStartedfindSubmittedfindTerminatedgetConfiggetErrorMessagesgetJobgetJobIdsgetJobInfogetJobLocationgetJobNrgetJobParamDfgetJobResourcesgetJobsgetLogFilesgetResourcesgetSSHWorkersInfogetStatusgrepLogsinstallPackagesOnSSHWorkerskillJobsloadConfigloadExportsloadRegistryloadResultloadResultsmakeClusterFunctionsmakeClusterFunctionsInteractivemakeClusterFunctionsLocalmakeClusterFunctionsLSFmakeClusterFunctionsMulticoremakeClusterFunctionsOpenLavamakeClusterFunctionsSGEmakeClusterFunctionsSLURMmakeClusterFunctionsSSHmakeClusterFunctionsTorquemakeJobmakeRegistrymakeSSHWorkermakeSubmitJobResultreduceResultsreduceResultsDataFramereduceResultsDataTablereduceResultsListreduceResultsMatrixreduceResultsVectorremoveRegistryremoveRegistryPackagesremoveRegistrySourceDirsremoveRegistrySourceFilesresetJobssanitizePathsetConfigsetJobFunctionsetJobNamessetRegistryPackagesshowClusterStatusshowLogshowStatussourceRegistryFilessubmitJobssweepRegistrysyncRegistrytestJobupdateRegistrywaitForJobs

Dependencies:backportsbase64encBBmiscbitbit64blobbrewcachemcheckmateclicpp11data.tableDBIdigestfastmapgluelifecyclememoisepkgconfigplogrrlangRSQLitesendmailRstringivctrs

Readme and manuals

Help Manual

Help pageTopics
Add packages to registry.addRegistryPackages
Add source dirs to registry.addRegistrySourceDirs
Add source files to registry.addRegistrySourceFiles
Map function over all combinations.batchExpandGrid
Export R object to be available on the slaves.batchExport
The BatchJobs packageBatchJobs
Maps a function over lists or vectors, adding jobs to a registry.batchMap
Combination of makeRegistry, batchMap and submitJobs.batchMapQuick
Maps a function over the results of a registry by using batchMap.batchMapResults
Manually query the BatchJobs databasebatchQuery
Reduces via a binary function over a list adding jobs to a registry.batchReduce
Reduces results via a binary function and adds jobs for this to a registry.batchReduceResults
Unload exported R objects.batchUnexport
Call an arbitrary function on specified SSH workers.callFunctionOnSSHWorkers
Cluster functions helper: Brew your template into a job description file.cfBrewTemplate
Cluster functions helper: Handle an unknown error during job submission.cfHandleUnknownSubmitError
Cluster functions helper: Kill a batch job via OS commandcfKillBatchJob
Cluster functions helper: Read in your brew template file.cfReadBrewTemplate
Check job ids.checkIds
BatchJobs configuration..BatchJobs.R configuration
Helper function to debug multicore mode.debugMulticore
Helper function to debug SSH mode.debugSSH
Find all results where a specific condition is true.filterResults
Find jobs depending on computional state.findDisappeared findDone findErrors findExpired findMissingResults findNotDone findNotErrors findNotOnSystem findNotRunning findNotStarted findNotSubmitted findNotTerminated findOnSystem findRunning findStarted findSubmitted findTerminated
Finds ids of jobs that match a query.findJobs
Returns a list of BatchJobs configuration settingsgetConfig
Get error messages of jobs.getErrorMessages
Get job from registry by id.getJob
Get ids of jobs in registry.getJobIds
Get computational information of jobs.getJobInfo
Get the physical location of job files on the hard disk.getJobLocation
Get number of jobs in registry.getJobNr
Retrieve Job Parameters.getJobParamDf
Function to get the resources that were submitted for some jobs.getJobResources
Get jobs from registry by id.getJobs
Get log file paths for jobs.getLogFiles
Function to get job resources in job function.getResources
Print and return R installation and other information for SSH workers.getSSHWorkersInfo
Grep log files for a pattern.grepLogs
Install packages on SSH workers.installPackagesOnSSHWorkers
Kill some jobs on the batch system.killJobs
Load a specific configuration file.loadConfig
Load exported R data objects.loadExports
Load a previously saved registry.loadRegistry
Loads a specific result file.loadResult
Loads result files for id vector.loadResults
Create a ClusterFuntions object.ClusterFunctions makeClusterFunctions
Create cluster functions for sequential execution in same session.makeClusterFunctionsInteractive
Create cluster functions for sequential execution on local host.makeClusterFunctionsLocal
Create cluster functions for LSF systems.makeClusterFunctionsLSF
Use multiple cores on local Linux machine to spawn parallel jobs.makeClusterFunctionsMulticore
Create cluster functions for OpenLava systems.makeClusterFunctionsOpenLava
Create cluster functions for Sun Grid Engine systems.makeClusterFunctionsSGE
Create cluster functions for SLURM-based systems.makeClusterFunctionsSLURM
Create an SSH cluster to execute jobs.makeClusterFunctionsSSH
Create cluster functions for torque-based systems.makeClusterFunctionsTorque
Creates a job description.Job makeJob
Construct a registry object.makeRegistry Registry
Create SSH worker for SSH cluster functions.makeSSHWorker SSHWorker
Create a SubmitJobResult object.makeSubmitJobResult SubmitJobResult
Reduce results from result directory.reduceResults reduceResultsDataFrame reduceResultsDataTable reduceResultsList reduceResultsMatrix reduceResultsVector
Remove a registry object.removeRegistry
Remove packages from registry.removeRegistryPackages
Remove packages from registry.removeRegistrySourceDirs
Remove source files from registry.removeRegistrySourceFiles
Reset computational state of jobs.resetJobs
Sanitize a pathsanitizePath
Set and overwrite configuration settingssetConfig
Sets the job function for already existing jobs.setJobFunction
Set job names.setJobNames
Set packages for a registry.setRegistryPackages
Show information about available computational resources on cluster.showClusterStatus
Display the contents of a log file.showLog
Retrieve or show status information about jobs.getStatus showStatus
Source registry filessourceRegistryFiles
Submit jobs or chunks of jobs to batch system via cluster function.submitJobs
Sweep obsolete files from the file system.sweepRegistry
Syncronize staged queries into the registry.syncRegistry
Tests a job by running it with Rscript in a new process.testJob
Wait for termination of jobs on the batch system.waitForJobs