diff --git a/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala b/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala index 8b117650ab5b9..25db71ca49132 100644 --- a/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala +++ b/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala @@ -329,21 +329,21 @@ private[yarn] class YarnAllocator( // to request YARN containers with extra resources without Spark scheduling on // them, the user can specify resources via the spark.yarn.executor.resource. // config. Those configs are only used in the base default profile though and do - // not get propogated into any other custom ResourceProfiles. This is because + // not get propagated into any other custom ResourceProfiles. This is because // there would be no way to remove them if you wanted a stage to not have them. // This results in your default profile getting custom resources defined in // spark.yarn.executor.resource. plus spark defined resources of // GPU or FPGA. Spark converts GPU and FPGA resources into the YARN built in // types yarn.io/gpu) and yarn.io/fpga, but does not // know the mapping of any other resources. Any other Spark custom resources - // are not propogated to YARN for the default profile. So if you want Spark + // are not propagated to YARN for the default profile. So if you want Spark // to schedule based off a custom resource and have it requested from YARN, you // must specify it in both YARN (spark.yarn.{driver/executor}.resource.) // and Spark (spark.{driver/executor}.resource.) configs. Leave the Spark // config off if you only want YARN containers with the extra resources but Spark not to // schedule using them. Now for custom ResourceProfiles, it doesn't currently have a way // to only specify YARN resources without Spark scheduling off of them. This means for - // custom ResourceProfiles we propogate all the resources defined in the ResourceProfile + // custom ResourceProfiles we propagate all the resources defined in the ResourceProfile // to YARN. We still convert GPU and FPGA to the YARN build in types as well. This requires // that the name of any custom resources you specify match what they are defined as in YARN. val customResources = if (rp.id == DEFAULT_RESOURCE_PROFILE_ID) {