设为首页收藏本站
开启辅助访问

Yarn REST API 使用指南

 您所在的位置：网站首页 › java代码杀掉yarn的任务 › Yarn REST API 使用指南

Yarn REST API 使用指南

2023-12-16 00:06| 来源: 网络整理| 查看: 265

Yarn作为一款优秀的开源集群管理工具，可以用它来运行Hadoop，Spark，Flink等大数据处理任务。所有的分布式计算框架，都是主从模式，ResourceManager作为集群的管理员，是任务提交的入口。一般企业的大数据处理平台会在Yarn的基础中做进一步的封装，以web应用的形式提供更高级的大数据处理平台。如果web应用通过shell的方式提交任务将显得太重，而且不利于任务状态的监控。幸好Yarn提供了ResourceManager的REST API，可以很方便的向集群提交、杀死任务或者监控任务的状态，本文将简要介绍ResourceManager REST API的使用方法。

向集群提交任务

先通过POST请求生成application_id再通过这个id来提交任务。

生成application-id

通过POST请求rm-http-address:port/ws/v1/cluster/apps/new-application，不需要参数，ResourceManager将返回application_id与集群可用资源配额，如下：

请求url

http://cdh-1:8088/ws/v1/cluster/apps/new-application

请求参数：无

返回数据

{ "application-id": "application_1613349389113_0535", "maximum-resource-capability": { "memory": 16384, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 16384 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } } }

返回json字段说明

ItemData TypeDescriptionapplication-idstringThe newly created application idmaximum-resource-capabilityobjectThe maximum resource capabilities available on this cluster

maximum-resource-capability元素说明

ItemData TypeDescriptionmemoryint容器可用的最大内存vCoresint容器可用的最大核心数提交任务

提交应用程序API可用于提交应用程序。如果提交申请，则必须首先使用Cluster New Application API获得一个应用程序ID 。应用程序ID必须是请求正文的一部分。响应包含指向应用程序页面的URL，可用于跟踪应用程序的状态和进度。

通过POST请求rm-http-address:port/ws/v1/cluster/apps提交任务

请求url

http://cdh-1:8088/ws/v1/cluster/apps

请求参数

POST http:///ws/v1/cluster/apps Accept: application/json Content-Type: application/json { "application-id":"application_1404203615263_0001", "application-name":"test", "am-container-spec": { "local-resources": { "entry": [ { "key":"AppMaster.jar", "value": { "resource":"hdfs://hdfs-namenode:9000/user/testuser/DistributedShell/demo-app/AppMaster.jar", "type":"FILE", "visibility":"APPLICATION", "size": 43004, "timestamp": 1405452071209 } } ] }, "commands": { "command":"{{JAVA_HOME}}/bin/java -Xmx10m org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster --container_memory 10 --container_vcores 1 --num_containers 1 --priority 0 1>/AppMaster.stdout 2>/AppMaster.stderr" }, "environment": { "entry": [ { "key": "DISTRIBUTEDSHELLSCRIPTTIMESTAMP", "value": "1405459400754" }, { "key": "CLASSPATH", "value": "{{CLASSPATH}}./*{{HADOOP_CONF_DIR}}{{HADOOP_COMMON_HOME}}/share/hadoop/common/*{{HADOOP_COMMON_HOME}}/share/hadoop/common/lib/*{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/*{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/lib/*{{HADOOP_YARN_HOME}}/share/hadoop/yarn/*{{HADOOP_YARN_HOME}}/share/hadoop/yarn/lib/*./log4j.properties" }, { "key": "DISTRIBUTEDSHELLSCRIPTLEN", "value": "6" }, { "key": "DISTRIBUTEDSHELLSCRIPTLOCATION", "value": "hdfs://hdfs-namenode:9000/user/testuser/demo-app/shellCommands" } ] } }, "unmanaged-AM":false, "max-app-attempts":2, "resource": { "memory":1024, "vCores":1 }, "application-type":"YARN", "keep-containers-across-application-attempts":false }

请求参数说明

ItemData TypeDescriptionapplication-idstring申请编号application-namestring申请名称queuestring应将应用程序提交到的队列的名称priorityint应用程序的优先级am-container-specobject应用程序主容器启动上下文，如下所述unmanaged-AMboolean该应用程序是否使用非托管应用程序主机max-app-attemptsint此应用程序的最大尝试次数resourceobject应用程序主机需要的资源，如下所述application-typestring应用程序类型（MapReduce，Pig，Hive等）keep-containers-across-application-attmptsbooleanYARN是否应保留此应用程序使用的容器而不是销毁它们application-tagsobject应用程序标签列表，请参阅有关如何指定标签的请求示例

am-container-spec对象的元素

应该使用am-container-spec对象为应用程序主机提供容器启动上下文。

ItemData TypeDescriptionlocal-resourcesobject描述需要本地化的资源的对象，如下所述environmentobject容器的环境变量，指定为键值对commandsobject用于启动容器的命令（应按执行顺序）service-dataobject特定于应用程序的服务数据；key是辅助服务的名称，值是您希望传递的数据的base-64编码credentialsobject您的应用程序运行所需的凭据，如下所述application-aclsobject您的应用程序的ACLs；密钥可以是“ VIEW_APP”或“ MODIFY_APP”，值是具有权限的用户列表

local-resource对象的元素

ItemData TypeDescriptionresourcestring要本地化的资源的位置typestring资源类型；选项是“ ARCHIVE”，“ FILE”和“ PATTERN”visibilitystring可见要本地化的资源；选项是“ PUBLIC”，“ PRIVATE”和“ APPLICATION”sizelong要本地化的资源大小timestamplong要本地化的资源的时间戳

credentials对象的元素

ItemData TypeDescriptiontokensobject您希望传递给应用程序的令牌，指定为键值对。密钥是令牌的标识符，值是令牌（应使用相应的Web服务获取）secretsobject您希望在应用程序中使用的机密，指定为键值对。它们的键是标识符，值是密钥的base-64编码

resource对象的元素

ItemData TypeDescriptionmemoryint每个容器所需的内存vCoresint每个容器所需的虚拟核心

返回数据： 无

提交后ResourceManager不会返回信息。从请求的参数可以看出，如果想通过Yarn来管理某个分布式计算任务，必须有对应的ApplicationMaser实现，如上例中的AppMaster.jar，其执行入口类为org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster。常见的分布式计算任务如Hadoop、Spark与Flink都有，但Flink-yarn项目对Flink 1.9貌似还 没有实现对应的ApplicationMaster，所以Flink 1.9的任务目前还不能通过Rest API提交（1.7可以）。

查询任务状态信息查询所有任务

通过GET请求rm-http-address:port/ws/v1/cluster/apps将获取到所有任务的信息列表，如

请求url

http://cdh-:8088/ws/v1/cluster/apps

请求参数（可选）

可以为GET操作指定多个参数。开始时间和结束时间都有一个begin和end参数，以允许您指定范围。例如，可以请求在2021年3月2日早上09:00:00和早上10:00：00之间启动的所有应用程序，其中startedTimeBegin = 1614646800000＆startedTimeEnd = 1614650400000。如果未指定Begin参数，则默认为0；如果未指定End参数，则默认为无穷大。

* state [deprecated] - state of the application * states - applications matching the given application states, specified as a comma-separated list.[NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] * finalStatus - the final status of the application - reported by the application itself [UNDEFINED, FAILED, KILLED, SUCCEEDED] * user - user name * queue - queue name * limit - total number of app objects to be returned * startedTimeBegin - applications with start time beginning with this time, specified in ms since epoch * startedTimeEnd - applications with start time ending with this time, specified in ms since epoch * finishedTimeBegin - applications with finish time beginning with this time, specified in ms since epoch * finishedTimeEnd - applications with finish time ending with this time, specified in ms since epoch * applicationTypes - applications matching the given application types, specified as a comma-separated list. * applicationTags - applications matching any of the given application tags, specified as a comma-separated list.

返回数据

{ "apps": { "app": [ { "id": "application_1613349389113_0393", "user": "root", "name": "com.zzmj.main.KafkaToParquet", "queue": "root.users.root", "state": "RUNNING", "finalStatus": "UNDEFINED", "progress": 10.0, "trackingUI": "ApplicationMaster", "trackingUrl": "http://cdh-1:8088/proxy/application_1613349389113_0393/", "diagnostics": "", "clusterId": 1613349389113, "applicationType": "SPARK", "applicationTags": "", "priority": 0, "startedTime": 1614648944707, "launchTime": 1614648945179, "finishedTime": 0, "elapsedTime": 515573848, "amContainerLogs": "http://cdh-4:8042/node/containerlogs/container_1613349389113_0393_01_000001/root", "amHostHttpAddress": "cdh-4:8042", "amRPCAddress": "cdh-4:46434", "allocatedMB": 5120, "allocatedVCores": 4, "reservedMB": 0, "reservedVCores": 0, "runningContainers": 2, "memorySeconds": 2514229328, "vcoreSeconds": 1914825, "queueUsagePercentage": 6.25, "clusterUsagePercentage": 6.25, "resourceSecondsMap": { "entry": { "key": "memory-mb", "value": "2514229328" }, "entry": { "key": "vcores", "value": "1914825" } }, "preemptedResourceMB": 0, "preemptedResourceVCores": 0, "numNonAMContainerPreempted": 0, "numAMContainerPreempted": 0, "preemptedMemorySeconds": 0, "preemptedVcoreSeconds": 0, "preemptedResourceSecondsMap": {}, "logAggregationStatus": "NOT_START", "unmanagedApplication": false, "amNodeLabelExpression": "", "timeouts": { "timeout": [ { "type": "LIFETIME", "expiryTime": "UNLIMITED", "remainingTimeInSeconds": -1 } ] } } ] } }

字段说明

ItemDataTypeDescriptionidstring应用的application-iduserstring提交任务的用户名namestring应用程序的名称queuestring应用程序所属消息队列statestring应用程序当前状态finalStatusstring应用程序最终状态progressdouble应用程序进度trackingUIstring追踪UI显示名称trackingUrlstring追踪UI的urlclusterIdstring集群idapplicationTypestring应用程序类型priorityint应用程序优先级startedTimelong应用程序开始时间launchTimelong应用程序加载时间finishedTimelong应用程序完成时间elapsedTimelong应用程序消耗时间(finished-start)amContainerLogsstringam容器日志地址amHostHttpAddressstringam的主机http地址amRPCAddressstringam的RPC地址allocatedMBstring初始化内存大小allocatedVCoresstring初始化核心数reservedMBstring保留内存reservedVCoresstring保留核心数runningContainersstring正在运行的容器数memorySecondsint所有的container每秒消耗的内存总和vcoreSecondstring所有的container每秒消耗的核心数总和queueUsagePercentagedouble所属队列的资源使用百分比clusterUsagePercentagedouble所属集群的资源使用百分比logAggregationStatusstring日志聚合状态unmanagedApplicationboolean未被管理的应用程序查询单个任务

通过GET请求rm-http-address:port/ws/v1/cluster/apps/{appid}，如

http://cdh-1:8088/ws/v1/cluster/apps/application_1613349389113_0001{ "app": { "id": "application_1613349389113_0001", "user": "root", "name": "StorageAgg", "queue": "root.users.root", "state": "RUNNING", "finalStatus": "UNDEFINED", "progress": 100.0, "trackingUI": "ApplicationMaster", "trackingUrl": "http://cdh-1:8088/proxy/application_1613349389113_0001/", "diagnostics": "", "clusterId": 1613349389113, "applicationType": "Apache Flink", "applicationTags": "", "priority": 0, "startedTime": 1613358478865, "launchTime": 1613358479810, "finishedTime": 0, "elapsedTime": 1829294258, "amContainerLogs": "http://cdh-4:8042/node/containerlogs/container_1613349389113_0001_01_000001/root", "amHostHttpAddress": "cdh-4:8042", "amRPCAddress": "cdh-4:8082", "allocatedMB": 9216, "allocatedVCores": 4, "reservedMB": 0, "reservedVCores": 0, "runningContainers": 2, "memorySeconds": 16858454105, "vcoreSeconds": 7317057, "queueUsagePercentage": 11.25, "clusterUsagePercentage": 11.25, "resourceSecondsMap": { "entry": { "key": "memory-mb", "value": "16858454105" }, "entry": { "key": "vcores", "value": "7317057" } }, "preemptedResourceMB": 0, "preemptedResourceVCores": 0, "numNonAMContainerPreempted": 0, "numAMContainerPreempted": 0, "preemptedMemorySeconds": 0, "preemptedVcoreSeconds": 0, "preemptedResourceSecondsMap": {}, "logAggregationStatus": "NOT_START", "unmanagedApplication": false, "amNodeLabelExpression": "", "timeouts": { "timeout": [ { "type": "LIFETIME", "expiryTime": "UNLIMITED", "remainingTimeInSeconds": -1 } ] } } }修改任务状态

通过Cluster Application State API来完成，当然需要得到RM web service的授权才能进行这种操作，URI为rm-http-address:port/ws/v1/cluster/apps/{appid}/state，如

通过GET请求该URI

http://cdh-1:8088/ws/v1/cluster/apps/application_1613349389113_0001/state

将返回的application_id为application_1613349389113_0001的任务的状态

{ "state": "RUNNING" }

然会参数列举说明

ItemData TypeDescriptionstatestringThe application state - can be one of “NEW”, “NEW_SAVING”, “SUBMITTED”, “ACCEPTED”, “RUNNING”, “FINISHED”, “FAILED”, “KILLED”

或者通过PUT请求来kill一个job

http://cdh-1:8088/ws/v1/cluster/apps/application_1399397633663_0003/state{ "state":"KILLED" }

返回结果为

{ "state":"RUNNING" }

其实的state的状态为当前程序的状态【RUNNING, ACCEPTED】

重新通过GET请求该任务的状态

发现该任务已经被kill了

{ "state":"KILLED" }查看与调整任务优先级

主要通过访问rm-http-address:port/ws/v1/cluster/apps/{appid}/priority来完成。统一需要得到RM web services的授权

查看任务优先级

通过GET请求上面的URI，如

http://cdh-1:8088/ws/v1/cluster/apps/application_1613349389113_0001/priority

将返回application_id为application_1613349389113_0001的任务的优先级

{ "priority": 0 }修改任务优先级

如果这项任务非常重要，你想让它优先执行，可通过PUT请求来修改其优先级

http://cdh-1:8088/ws/v1/cluster/apps/application_1613349389113_0001/priority 请求参数： Accept: application/json Content-Type: application/json { "priority": 8 }集群API集群信息

主要通过GET请求访问rm-http-address:port/ws/v1/cluster来完成。

通过GET请求上面的URI，如

http://cdh-1:8088/ws/v1/cluster

请求参数： 无

返回数据

{ "clusterInfo": { "id": 1613349389113, "startedOn": 1613349389113, "state": "STARTED", "haState": "ACTIVE", "rmStateStoreName": "org.apache.hadoop.yarn.server.resourcemanager.recovery.NullRMStateStore", "resourceManagerVersion": "3.0.0-cdh6.3.2", "resourceManagerBuildVersion": "3.0.0-cdh6.3.2 from 9aff20de3b5ecccf3c19d57f71b214fb4d37ee89 by jenkins source checksum bdea11f98ce3d056d6c170c883b73569", "resourceManagerVersionBuiltOn": "2019-11-08T13:55Z", "hadoopVersion": "3.0.0-cdh6.3.2", "hadoopBuildVersion": "3.0.0-cdh6.3.2 from 9aff20de3b5ecccf3c19d57f71b214fb4d37ee89 by jenkins source checksum f539c87da37534aad732f2a7ddcc59", "hadoopVersionBuiltOn": "2019-11-08T13:49Z", "haZooKeeperConnectionState": "Could not find leader elector. Verify both HA and automatic failover are enabled." } }

返回数据字段说明

ItemData TypeDescriptionidlong集群IDstartedOnlong集群启动的时间（从纪元开始以毫秒为单位）statestringResourceManager状态-有效值为：NOTINITED，INITED，STARTED，STOPPEDhaStatestringResourceManager HA状态-有效值为：INITIALIZING，ACTIVE，STANDBY，STOPPEDrmStateStoreNamestring实现ResourceManager状态存储的类的完全限定名称resourceManagerVersionstringResourceManager的版本resourceManagerBuildVersionstringResourceManager构建字符串以及构建版本，用户和校验和resourceManagerVersionBuiltOnstring生成ResourceManager的时间戳（自纪元以来以毫秒为单位）hadoopVersionstringHadoop通用版本hadoopBuildVersionstring具有构建版本，用户和校验和的Hadoop通用构建字符串hadoopVersionBuiltOnstring建立hadoop common的时间戳（自纪元以来以毫秒为单位）haZooKeeperConnectionStatestringZooKeeper高可用性服务的连接状态集群指标

通过GET请求rm-http-address:port/ws/v1/cluster/metrics来完成。

通过GET请求上面URI，如

http://cdh-1:8088/ws/v1/cluster/metrics

请求参数： 无

返回数据

{ "clusterMetrics": { "appsSubmitted": 547, "appsCompleted": 506, "appsPending": 0, "appsRunning": 8, "appsFailed": 14, "appsKilled": 19, "reservedMB": 0, "availableMB": 36864, "allocatedMB": 45056, "reservedVirtualCores": 0, "availableVirtualCores": 8, "allocatedVirtualCores": 32, "containersAllocated": 16, "containersReserved": 0, "containersPending": 0, "totalMB": 81920, "totalVirtualCores": 40, "totalNodes": 5, "lostNodes": 0, "unhealthyNodes": 0, "decommissioningNodes": 0, "decommissionedNodes": 0, "rebootedNodes": 0, "activeNodes": 5, "shutdownNodes": 0 } }

返回数据字段说明

ItemData TypeDescriptionappsSubmittedint提交的应用程序数量appsCompletedint完成的应用程序数量appsPendingint等待的应用程序数量appsRunningint正在运行的应用程序数量appsFailedint失败的应用程序数量appsKilledint被杀死的应用程序数量reservedMBlong保留的内存量（MB）availableMBlong可用的内存量（MB）allocatedMBlong分配的内存量（MB）totalMBlong总内存量（MB）reservedVirtualCoreslong保留的虚拟核心数availableVirtualCoreslong可用虚拟核心数allocatedVirtualCoreslong分配的虚拟核心数totalVirtualCoreslong虚拟核心总数containersAllocatedint分配的容器数containersReservedint保留的容器数containersPendingint待处理的容器数totalNodesint节点总数activeNodesint活动节点数lostNodesint丢失的节点数unhealthyNodesint不良节点数decommissioningNodesint停用的节点数decommissionedNodesint退役的节点数rebootedNodesint重新启动的节点数shutdownNodesint关闭的节点数集群调度

调度程序资源包含有关集群中配置的当前调度程序的信息。它目前支持Fifo，容量和公平调度程序。根据配置哪个调度程序，您将获得不同的信息，因此请务必查看类型信息。

请求URI

http://cdh-1:8088/ws/v1cluster/scheduler

请求参数： 无

返回数据

{ "scheduler": { "schedulerInfo": { "type": "fairScheduler", "rootQueue": { "maxApps": 2147483647, "minResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 16384, "minimumAllocation": 1024, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 8, "minimumAllocation": 1, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "usedResources": { "memory": 45056, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 45056 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "amUsedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "amMaxResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "demandResources": { "memory": 45056, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 45056 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "steadyFairResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "fairResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "clusterResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "reservedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxContainerAllocation": { "memory": 16384, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 16384 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } }, "allocatedContainers": 16, "reservedContainers": 0, "queueName": "root", "schedulingPolicy": "DRF", "preemptable": true, "childQueues": { "queue": [ { "type": "fairSchedulerLeafQueueInfo", "maxApps": 2147483647, "minResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 16384, "minimumAllocation": 1024, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 8, "minimumAllocation": 1, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "usedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "amUsedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "amMaxResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "demandResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "steadyFairResources": { "memory": 40960, "vCores": 20, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 40960 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 20 } ] } }, "fairResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "clusterResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "reservedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxContainerAllocation": { "memory": 16384, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 16384 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } }, "allocatedContainers": 0, "reservedContainers": 0, "queueName": "root.default", "schedulingPolicy": "DRF", "preemptable": true, "numPendingApps": 0, "numActiveApps": 0 }, { "maxApps": 2147483647, "minResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 16384, "minimumAllocation": 1024, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 8, "minimumAllocation": 1, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "usedResources": { "memory": 45056, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 45056 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "amUsedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "amMaxResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "demandResources": { "memory": 45056, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 45056 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "steadyFairResources": { "memory": 40960, "vCores": 20, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 40960 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 20 } ] } }, "fairResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "clusterResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "reservedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxContainerAllocation": { "memory": 16384, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 16384 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } }, "allocatedContainers": 16, "reservedContainers": 0, "queueName": "root.users", "schedulingPolicy": "DRF", "preemptable": true, "childQueues": { "queue": [ { "type": "fairSchedulerLeafQueueInfo", "maxApps": 2147483647, "minResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 16384, "minimumAllocation": 1024, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 8, "minimumAllocation": 1, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxResources": { "memory": 65536, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 65536 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "usedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "amUsedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "amMaxResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "demandResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "steadyFairResources": { "memory": 13654, "vCores": 7, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 13654 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 7 } ] } }, "fairResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "clusterResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "reservedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxContainerAllocation": { "memory": 16384, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 16384 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } }, "allocatedContainers": 0, "reservedContainers": 0, "queueName": "root.users.hdfs", "schedulingPolicy": "DRF", "preemptable": true, "numPendingApps": 0, "numActiveApps": 0 }, { "type": "fairSchedulerLeafQueueInfo", "maxApps": 2147483647, "minResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 16384, "minimumAllocation": 1024, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 8, "minimumAllocation": 1, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "usedResources": { "memory": 45056, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 45056 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "amUsedResources": { "memory": 9728, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 9728 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } }, "amMaxResources": { "memory": 40960, "vCores": 20, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 40960 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 20 } ] } }, "demandResources": { "memory": 45056, "vCores": 32, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 45056 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 32 } ] } }, "steadyFairResources": { "memory": 13654, "vCores": 7, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 13654 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 7 } ] } }, "fairResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "clusterResources": { "memory": 81920, "vCores": 40, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 81920 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 40 } ] } }, "reservedResources": { "memory": 0, "vCores": 0, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 0 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 0 } ] } }, "maxContainerAllocation": { "memory": 16384, "vCores": 8, "resourceInformations": { "resourceInformation": [ { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "memory-mb", "resourceType": "COUNTABLE", "units": "Mi", "value": 16384 }, { "maximumAllocation": 9223372036854775807, "minimumAllocation": 0, "name": "vcores", "resourceType": "COUNTABLE", "units": "", "value": 8 } ] } }, "allocatedContainers": 16, "reservedContainers": 0, "queueName": "root.users.root", "schedulingPolicy": "fair", "preemptable": true, "numPendingApps": 0, "numActiveApps": 8 } ] } } ] } } } } }集群应用程序尝试API

使用应用程序尝试API，您可以获得代表应用程序尝试的资源的集合。在此资源上运行GET操作时，将获得App Attempt对象的集合。

请求URI

http://cdh-1:8088/ws/v1/cluster/apps/{appid}/appattempts

请求参数： 无

返回数据

{ "appAttempts": { "appAttempt": [ { "id": 1, "startTime": 1615272507648, "finishedTime": 0, "containerId": "container_1615272298938_0001_01_000001", "nodeHttpAddress": "cdh-6:8042", "nodeId": "cdh-6:8041", "logsLink": "http://cdh-6:8042/node/containerlogs/container_1615272298938_0001_01_000001/root", "blacklistedNodes": "", "nodesBlacklistedBySystem": "", "appAttemptId": "appattempt_1615272298938_0001_000001" } ] } }ItemData TypeDescriptionidstring应用尝试IDnodeIdstring尝试运行的节点的节点IDnodeHttpAddressstring尝试运行的节点的节点http地址logsLinkstring应用程序尝试日志的http链接containerIdstring应用尝试容器的IDstartTimelong尝试的开始时间（自纪元以来以毫秒为单位）


【本文地址】


今日新闻

推荐新闻

CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3