pyspark.resource.TaskResourceRequests#
- class pyspark.resource.TaskResourceRequests(_jvm=None, _requests=None)[source]#
A set of task resource requests. This is used in conjunction with the
pyspark.resource.ResourceProfileBuilderto programmatically specify the resources needed for an RDD that will be applied at the stage level.New in version 3.1.0.
See also
Notes
This API is evolving.
Methods
cpus(amount)Specify number of cpus per Task.
resource(resourceName, amount)Amount of a particular custom resource(GPU, FPGA, etc) to use.
Attributes
requests- Returns