Back to Hadoop

Licensed to the Apache Software Foundation (ASF) under one

hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha2/CHANGELOG.3.0.0-alpha2.md

2.0.5-alpha-rc2154.6 KB
Original Source
<!--- # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -->

Apache Hadoop Changelog

Release 3.0.0-alpha2 - 2017-01-25

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-13361Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)MajorscriptsAllen WittenauerAllen Wittenauer
HDFS-6962ACL inheritance conflicts with umaskmodeCriticalsecurityLINTEJohn Zhuge
HADOOP-13341Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTSMajorscriptsAllen WittenauerAllen Wittenauer
HADOOP-13588ConfServlet should respect Accept request headerMajorconfWeiwei YangWeiwei Yang
HDFS-10636Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.Majordatanode, fsVirajith JalapartiVirajith Jalaparti
HADOOP-13218Migrate other Hadoop side tests to prepare for removing WritableRPCEngineMajortestKai ZhengWei Zhou
HDFS-10877Make RemoteEditLogManifest.committedTxnId optional in Protocol BuffersMajorqjmSean MackrorySean Mackrory
HADOOP-13681Reduce Kafka dependencies in hadoop-kafka moduleMajormetricsGrant HenkeGrant Henke
HADOOP-13678Update jackson from 1.9.13 to 2.x in hadoop-toolsMajortoolsAkira AjisakaAkira Ajisaka
MAPREDUCE-6776yarn.app.mapreduce.client.job.max-retries should have a more useful defaultMajorclientDaniel TempletonMiklos Szegedi
HADOOP-13699Configuration does not substitute multiple references to the same varCriticalconfAndrew WangAndrew Wang
HDFS-10637Modifications to remove the assumption that FsVolumes are backed by java.io.File.Majordatanode, fsVirajith JalapartiVirajith Jalaparti
HDFS-10916Switch from "raw" to "system" xattr namespace for erasure coding policyMajorerasure-codingAndrew WangAndrew Wang
YARN-4464Lower the default max applications stored in the RM and storeBlockerresourcemanagerKWON BYUNGCHANGDaniel Templeton
HADOOP-13721Remove stale method ViewFileSystem#getTrashCanLocationMinorviewfsManoj GovindassamyManoj Govindassamy
HDFS-10957Retire BKJM from trunkMajorhaVinayakumar BVinayakumar B
YARN-5718TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behaviorMajorresourcemanager, timelineclientJunping DuJunping Du
HADOOP-13560S3ABlockOutputStream to support huge (many GB) file writesMajorfs/s3Steve LoughranSteve Loughran
MAPREDUCE-6791remove unnecessary dependency from hadoop-mapreduce-client-jobclient to hadoop-mapreduce-client-shuffleMinormrv2Haibo ChenHaibo Chen
HADOOP-7352FileSystem#listStatus should throw IOE upon access errorMajorfsMatt FoleyJohn Zhuge
HADOOP-13693Remove the message about HTTP OPTIONS in SPNEGO initialization message from kms audit logMinorkmsXiao ChenXiao Chen
YARN-5388Deprecate and remove DockerContainerExecutorCriticalnodemanagerDaniel TempletonDaniel Templeton
YARN-3732Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classesMinor.Devaraj KDevaraj K
HDFS-11048Audit Log should escape control charactersMajor.Eric BadgerEric Badger
HADOOP-13792Stackoverflow for schemeless defaultFS with trailing slashMajorfsDarius MurawskiJohn Zhuge
HDFS-10970Update jackson from 1.9.13 to 2.x in hadoop-hdfsMajorbuildAkira AjisakaAkira Ajisaka
HDFS-11116Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValueMinor.Yiqun LinYiqun Lin
HADOOP-12718Incorrect error message by fs -put local dir without permissionMajor.John ZhugeJohn Zhuge
HADOOP-13660Upgrade commons-configuration version to 2.1MajorbuildSean MackrorySean Mackrory
HADOOP-12705Upgrade Jackson 2.2.3 to 2.7.8MajorbuildSteve LoughranSean Mackrory
YARN-5713Update jackson from 1.9.13 to 2.x in hadoop-yarnMajorbuild, timelineserverAkira AjisakaAkira Ajisaka
HADOOP-13050Upgrade to AWS SDK 1.11.45Blockerbuild, fs/s3Steve LoughranSteve Loughran
HADOOP-13706Update jackson from 1.9.13 to 2.x in hadoop-common-projectMajorbuildAkira AjisakaAkira Ajisaka
HADOOP-13812Upgrade Tomcat to 6.0.48BlockerkmsJohn ZhugeJohn Zhuge
HDFS-5517Lower the default maximum number of blocks per fileMajornamenodeAaron T. MyersAaron T. Myers
HADOOP-13842Update jackson from 1.9.13 to 2.x in hadoop-maven-pluginsMinorbuildAkira AjisakaAkira Ajisaka
MAPREDUCE-4683Create and distribute hadoop-mapreduce-client-core-tests.jarCriticalbuildArun C MurthyAkira Ajisaka
HADOOP-13597Switch KMS from Tomcat to JettyMajorkmsJohn ZhugeJohn Zhuge
YARN-6071Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only)Blocker.Junping DuWangda Tan
HADOOP-13964Remove vestigal templates directories creationMajorbuildAllen WittenauerAllen Wittenauer

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-12756Incorporate Aliyun OSS file system implementationMajorfs, fs/ossshimingfeimingfei.shi
MAPREDUCE-6774Add support for HDFS erasure code policy to TestDFSIOMajor.Sammi ChenSammi Chen
HDFS-10918Add a tool to get FileEncryptionInfo from CLIMajorencryptionXiao ChenXiao Chen
HADOOP-13584hadoop-aliyun: merge HADOOP-12756 branch backMajorfsshimingfeiGenmao Yu
HDFS-9820Improve distcp to support efficient restore to an earlier snapshotMajordistcpYongjun ZhangYongjun Zhang
HADOOP-13716Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setupMajortestSteve LoughranSteve Loughran
YARN-4597Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycleMajornodemanagerChris DouglasArun Suresh
HADOOP-13852hadoop build to allow hadoop version property to be explicitly setMinorbuildSteve LoughranSteve Loughran
HADOOP-13578Add Codec for ZStandard CompressionMajor.churro moraleschurro morales
HADOOP-13933Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagersMajortoolsSurendra Singh LilhoreSurendra Singh Lilhore
HADOOP-13673Update scripts to be smarter when running with privilegeMajorscriptsAllen WittenauerAllen Wittenauer
YARN-2877Extend YARN to support distributed schedulingMajornodemanager, resourcemanagerSriram RaoKonstantinos Karanasos

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HDFS-10814Add assertion for getNumEncryptionZones when no EZ is createdMinortestVinitha Reddy GankidiVinitha Reddy Gankidi
HDFS-10784Implement WebHdfsFileSystem#listStatusIteratorMajorwebhdfsAndrew WangAndrew Wang
HDFS-10817Add Logging for Long-held NN Read LocksMajorlogging, namenodeErik KrogenErik Krogen
HADOOP-13465Design Server.Call to be extensible for unified call queueMajoripcDaryn SharpDaryn Sharp
HDFS-10822Log DataNodes in the write pipelineTrivialhdfs-clientJohn ZhugeJohn Zhuge
HDFS-10833Fix JSON errors in WebHDFS.md examplesTrivialdocumentationAndrew WangAndrew Wang
YARN-5616Clean up WeightAdjusterMajorfairschedulerYufei GuYufei Gu
HDFS-10778Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewerMajortoolsYiqun LinYiqun Lin
HDFS-10847Complete the document for FileDistribution processor in OfflineImageViewerMinordocumentationYiqun LinYiqun Lin
HADOOP-13519Make Path serializableMinorioSteve LoughranSteve Loughran
HDFS-10742Measure lock time in FsDatasetImplMajordatanodeChen LiangChen Liang
HDFS-10831Add log when URLConnectionFactory.openConnection failedMinorwebhdfsyunjiong zhaoyunjiong zhao
HDFS-10855Fix typos in HDFS documentsMinordocumentationYiqun LinYiqun Lin
HDFS-10837Standardize serializiation of WebHDFS DirectoryListingMajorwebhdfsAndrew WangAndrew Wang
HADOOP-13598Add eol=lf for unix format files in .gitattributesMajor.Akira AjisakaYiqun Lin
HADOOP-13411Checkstyle suppression by annotation or commentMinor.John ZhugeJohn Zhuge
HADOOP-13412Move dev-support/checkstyle/suppressions.xml to hadoop-build-toolsTrivial.John ZhugeJohn Zhuge
HADOOP-13580If user is unauthorized, log "unauthorized" instead of "Invalid signed text:"MinorsecurityWei-Chiu ChuangWei-Chiu Chuang
HDFS-10823Implement HttpFSFileSystem#listStatusIteratorMajorhttpfsAndrew WangAndrew Wang
HDFS-10489Deprecate dfs.encryption.key.provider.uri for HDFS encryption zonesMinor.Xiao ChenXiao Chen
HDFS-10868Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLEDTrivial.Andrew WangAndrew Wang
YARN-5540scheduler spends too much time looking at empty prioritiesMajorcapacity scheduler, fairscheduler, resourcemanagerNathan RobertsJason Lowe
HDFS-10875Optimize du -x to cache intermediate resultMajorsnapshotsXiao ChenXiao Chen
YARN-4591YARN Web UIs should provide a robots.txtTrivial.Lars FranckeSidharta Seethana
MAPREDUCE-6632Master.getMasterAddress() should be updated to use YARN-4629MinorapplicationmasterDaniel TempletonDaniel Templeton
YARN-5622TestYarnCLI.testGetContainers fails due to mismatched date formatsMinor.Eric BadgerEric Badger
HDFS-10876Dispatcher#dispatch should log IOException stacktraceTrivialbalancer & moverWei-Chiu ChuangManoj Govindassamy
YARN-3692Allow REST API to set a user generated message when killing an applicationMajor.Rajat JainRohith Sharma K S
HDFS-10869Remove the unused method InodeId#checkId()MajornamenodeJagadesh Kiran NJagadesh Kiran N
YARN-3877YarnClientImpl.submitApplication swallows exceptionsMinorclientSteve LoughranVarun Saxena
HADOOP-13658Replace config key literal strings with config key names I: hadoop commonMinorconfChen LiangChen Liang
YARN-5400Light cleanup in ZKRMStateStoreTrivialresourcemanagerDaniel TempletonDaniel Templeton
MAPREDUCE-6718add progress log to JHS during startupMinorjobhistoryserverHaibo ChenHaibo Chen
HADOOP-13537Support external calls in the RPC call queueMajoripcDaryn SharpDaryn Sharp
HDFS-10908Improve StripedBlockReader#createBlockReader error loggingMinorerasure-codingWei-Chiu ChuangManoj Govindassamy
HDFS-10910HDFS Erasure Coding doc should state its currently supported erasure coding policiesMajordocumentation, erasure-codingWei-Chiu ChuangYiqun Lin
HADOOP-13317Add logs to KMS server-side to improve supportabilityMinorkmsXiao ChenSuraj Acharya
HDFS-10940Reduce performance penalty of block caching when not usedMajornamenodeDaryn SharpDaryn Sharp
MAPREDUCE-6638Do not attempt to recover progress from previous job attempts if spill encryption is enabledMajorapplicationmasterKarthik KambatlaHaibo Chen
YARN-4855Should check if node exists when replace nodelabelsMinor.Tao JieTao Jie
HDFS-10690Optimize insertion/removal of replica in ShortCircuitCacheMajorhdfs-clientFenghua HuFenghua Hu
HADOOP-13685Document -safely option of rm command.Major.Xiaoyu YaoXiaoyu Yao
HADOOP-13628Support to retrieve specific property from configuration via REST APIMajorconfWeiwei YangWeiwei Yang
HDFS-10683Make class Token$PrivateToken privateMinor.John ZhugeJohn Zhuge
HDFS-10963Reduce log level when network topology cannot find enough datanodes.Minor.Xiao ChenXiao Chen
HADOOP-13323Downgrade stack trace on FS load from Warn to debugMinorfsSteve LoughranSteve Loughran
HADOOP-13688Stop bundling HTML source code in javadoc JARsMajorbuildAndrew WangAndrew Wang
HADOOP-13150Avoid use of toString() in output of HDFS ACL shell commands.Minor.Chris NaurothChris Nauroth
HADOOP-13689Do not attach javadoc and sources jars during non-dist buildMajor.Andrew WangAndrew Wang
HADOOP-12579Deprecate WriteableRPCEngineMajor.Haohui MaiWei Zhou
HADOOP-13641Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentationMinor.Xiao ChenHuafeng Wang
YARN-5551Ignore file backed pages from memory computation when smaps is enabledMinor.Rajesh BalamohanRajesh Balamohan
HDFS-10933Refactor TestFsckMinor.Takanobu AsanumaTakanobu Asanuma
HADOOP-13684Snappy may complain Hadoop is built without snappy if libhadoop is not found.Minor.Wei-Chiu ChuangWei-Chiu Chuang
HADOOP-13705Revert HADOOP-13534 Remove unused TrashPolicy#getInstance and initialize codeMajor.Andrew WangAndrew Wang
HADOOP-13698Document caveat for KeyShell when underlying KeyProvider does not delete a keyMinordocumentation, kmsXiao ChenXiao Chen
HDFS-10789Route webhdfs through the RPC call queueMajoripc, webhdfsDaryn SharpDaryn Sharp
HDFS-10903Replace config key literal strings with config key names II: hadoop hdfsMinor.Mingliang LiuChen Liang
HADOOP-13710Supress CachingGetSpaceUsed from logging interrupted exception stacktraceMinorfsWei-Chiu ChuangHanisha Koneru
YARN-5599Publish AM launch command to ATSMajor.Daniel TempletonRohith Sharma K S
HDFS-11012Unnecessary INFO logging on DFSClients for InvalidTokenMinorfsHarsh JHarsh J
HDFS-11003Expose "XmitsInProgress" through DataNodeMXBeanMajor.Brahma Reddy BattulaBrahma Reddy Battula
HADOOP-13661Upgrade HTrace versionMajor.Sean MackrorySean Mackrory
HADOOP-13722Code cleanup -- ViewFileSystem and InodeTreeMinorviewfsManoj GovindassamyManoj Govindassamy
HADOOP-13724Fix a few typos in site markdown documentsMinordocumentationAndrew WangDing Fei
YARN-5466DefaultContainerExecutor needs JavaDocsMinornodemanagerDaniel TempletonDaniel Templeton
HADOOP-13522Add %A and %a formats for fs -stat command to print permissionsMajorfsAlex GarbariniAlex Garbarini
HDFS-11009Add a tool to reconstruct block meta file from CLIMajordatanodeXiao ChenXiao Chen
HDFS-9480Expose nonDfsUsed via StorageTypeStatsMajor.Brahma Reddy BattulaBrahma Reddy Battula
HADOOP-13737Cleanup DiskChecker interfaceMajorutilArpit AgarwalArpit Agarwal
HADOOP-13702Add a new instrumented read-write lockMajorcommonJingcheng DuJingcheng Du
HADOOP-13732Upgrade OWASP dependency-check plugin versionMinorsecurityMike YoderMike Yoder
HADOOP-12082Support multiple authentication schemes via AuthenticationFilterMajorsecurityHrishikesh GadreHrishikesh Gadre
HADOOP-13669KMS Server should log exceptions before throwingMajorkmsXiao ChenSuraj Acharya
MAPREDUCE-6792Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir()MajorclientSanthosh G NayakSanthosh G Nayak
HDFS-5684Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as VisibleForTestingMinorhdfs-clientKeith TurnerManoj Govindassamy
YARN-5575Many classes use bare yarn. properties instead of the defined constantsMajor.Daniel TempletonDaniel Templeton
HADOOP-13659Upgrade jaxb-api versionMajorbuildSean MackrorySean Mackrory
HADOOP-13502Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests.MinortestChris NaurothChris Nauroth
HDFS-11049The description of dfs.block.replicator.classname is not clearMinordocumentationYiqun LinYiqun Lin
HDFS-11069Tighten the authorization of datanode RPCMajordatanode, securityKihwal LeeKihwal Lee
YARN-4456Clean up Lint warnings in nodemanagerMinornodemanagerDaniel TempletonDaniel Templeton
HDFS-11055Update default-log4j.properties for httpfs to imporve test loggingMajorhttpfs, testWei-Chiu ChuangWei-Chiu Chuang
YARN-4710Reduce logging application reserved debug info in FSAppAttempt#assignContainerMinorfairschedulerYiqun LinYiqun Lin
YARN-4668Reuse objectMapper instance in YarnMajortimelineclientYiqun LinYiqun Lin
HDFS-11064Mention the default NN rpc ports in hdfs-default.xmlMinordocumentationAndrew WangYiqun Lin
HDFS-10926Update staled configuration properties related to erasure codingMajor.Sammi ChenSammi Chen
YARN-4963capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurableMajorcapacityschedulerNathan RobertsNathan Roberts
HDFS-11047Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNodeMajordatanodeXiaobing ZhouXiaobing Zhou
HADOOP-10075Update jetty dependency to version 9Critical.Robert RatiRobert Kanter
MAPREDUCE-6799Document mapreduce.jobhistory.webapp.https.address in mapred-default.xmlMinordocumentation, jobhistoryserverAkira AjisakaYiqun Lin
HDFS-11074Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorageMajordatanodeArpit AgarwalHanisha Koneru
YARN-4907Make all MockRM#waitForState consistent.MajorresourcemanagerYufei GuYufei Gu
YARN-4396Log the trace information on FSAppAttempt#assignContainerMajorapplications, fairschedulerYiqun LinYiqun Lin
MAPREDUCE-6795Update the document for JobConf#setNumReduceTasksMajordocumentationAkira AjisakaYiqun Lin
HADOOP-13603Ignore package line length checkstyle ruleMajorbuildShane KumpfShane Kumpf
HADOOP-13583Incorporate checkcompatibility script which runs Java API Compliance CheckerMajorscriptsAndrew WangAndrew Wang
HADOOP-13667Fix typing mistake of inline document in hadoop-metrics2.propertiesMajordocumentationRui GaoRui Gao
HDFS-10909De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReadsMinor.Wei-Chiu ChuangManoj Govindassamy
HADOOP-13784Output javadoc inside the target directoryMajordocumentationAndrew WangAndrew Wang
HDFS-11080Update HttpFS to use ConfigRedactorMajor.Sean MackrorySean Mackrory
YARN-5697Use CliParser to parse options in RMAdminCLIMajorresourcemanagerTao JieTao Jie
HADOOP-12453Support decoding KMS Delegation Token with its own IdentifierMajorkms, securityXiaoyu YaoXiaoyu Yao
HDFS-11088Quash unnecessary safemode WARN message during NameNode startupTrivial.Andrew WangYiqun Lin
MAPREDUCE-6796Remove unused properties from JTConfig.javaMajor.Akira AjisakaHaibo Chen
YARN-4998Minor cleanup to UGI use in AdminServiceTrivialresourcemanagerDaniel TempletonDaniel Templeton
HDFS-10756Expose getTrashRoot to HTTPFS and WebHDFSMajorencryption, httpfs, webhdfsXiao ChenYuanbo Liu
MAPREDUCE-6790Update jackson from 1.9.13 to 2.x in hadoop-mapreduceMajorbuildAkira AjisakaAkira Ajisaka
YARN-5720Update document for "rmadmin -replaceLabelOnNode"Minor.Tao JieTao Jie
YARN-5356NodeManager should communicate physical resource capability to ResourceManagerMajornodemanager, resourcemanagerNathan RobertsÍñigo Goiri
HADOOP-13802Make generic options help more consistent, and alignedMinor.Grant SohnGrant Sohn
HADOOP-13782Make MutableRates metrics thread-local write, aggregate-on-readMajormetricsErik KrogenErik Krogen
HDFS-9482Replace DatanodeInfo constructors with a builder patternMajor.Brahma Reddy BattulaBrahma Reddy Battula
HADOOP-13800Remove unused HADOOP_AUDIT_LOGGER from hadoop-env.shMinorscriptsAkira AjisakaYiqun Lin
HADOOP-13590Retry until TGT expires even if the UGI renewal thread encountered exceptionMajorsecurityXiao ChenXiao Chen
HDFS-11120TestEncryptionZones should waitActiveMinortestXiao ChenJohn Zhuge
HADOOP-13687Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with Hadoop.MajorbuildChris NaurothChris Nauroth
YARN-5552Add Builder methods for common yarn API recordsMajor.Arun SureshTao Jie
HDFS-10941Improve BlockManager#processMisReplicatesAsync logMajornamenodeXiaoyu YaoChen Liang
YARN-4033In FairScheduler, parent queues should also display queue statusMajorfairschedulerSiqi LiSiqi Li
HADOOP-13810Add a test to verify that Configuration handles &-encoded charactersMinortestSteve LoughranSteve Loughran
HADOOP-13427Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}MajorfsSteve LoughranMingliang Liu
YARN-5736YARN container executor config does not handle white spaceTrivialnodemanagerMiklos SzegediMiklos Szegedi
YARN-5375invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failuresMajorresourcemanagersandfleesandflee
HDFS-11147Remove confusing log output in FsDatasetImpl#getInitialVolumeFailureInfosMinordatanodeChen LiangChen Liang
HADOOP-13742Expose "NumOpenConnectionsPerUser" as a metricMajor.Brahma Reddy BattulaBrahma Reddy Battula
HDFS-11117Refactor striped file tests to allow flexibly test erasure coding policyMajor.Sammi ChenSammi Chen
HADOOP-13646Remove outdated overview.htmlMinor.Akira AjisakaBrahma Reddy Battula
YARN-5722FairScheduler hides group resolution exceptions when assigning queueMajorfairschedulerWilfred SpiegelenburgWilfred Spiegelenburg
HADOOP-13166add getFileStatus("/") test to AbstractContractGetFileStatusTestMinorfs, testSteve LoughranSteve Loughran
HADOOP-11552Allow handoff on the server side for RPC requestsMajoripcSiddharth SethSiddharth Seth
HADOOP-13605Clean up FileSystem javadocs, logging; improve diagnostics on FS loadMajorfsSteve LoughranSteve Loughran
HADOOP-13018Make Kdiag check whether hadoop.token.files points to existent and valid filesMajor.Ravi PrakashRavi Prakash
HADOOP-11603Metric Snapshot log can be changed #MetricsSystemImpl.java since all the services will be initializedMinormetricsBrahma Reddy BattulaBrahma Reddy Battula
HADOOP-10776Open up already widely-used APIs for delegation-token fetching & renewal to ecosystem projectsBlocker.Robert Joseph EvansVinod Kumar Vavilapalli
HDFS-11175Document uppercase key names are not supported in TransparentEncryption.mdMinordocumentationYuanbo LiuYiqun Lin
HADOOP-1381The distance between sync blocks in SequenceFiles should be configurableMajorioOwen O'MalleyHarsh J
HADOOP-13506Redundant groupid warning in child projectsMinor.Kai SasakiKai Sasaki
YARN-5890FairScheduler should log information about AM-resource-usage and max-AM-share for queuesMajorfairschedulerYufei GuYufei Gu
YARN-4997Update fair scheduler to use pluggable auth providerMajorfairschedulerDaniel TempletonTao Jie
HADOOP-13790Make qbt script executableTrivialscriptsAndrew WangAndrew Wang
HDFS-8674Improve performance of postponed block scansCriticalnamenodeDaryn SharpDaryn Sharp
MAPREDUCE-6787Allow job_conf.xml to be downloadable on the job overview page in JHSMajorjobhistoryserverHaibo ChenHaibo Chen
HDFS-10581Hide redundant table on NameNode WebUI when no nodes are decomissioningTrivialhdfs, uiWeiwei YangWeiwei Yang
HDFS-11211Add a time unit to the DataNode client trace formatMinordatanodeAkira AjisakaJagadesh Kiran N
HADOOP-13827Add reencryptEncryptedKey interface to KMSMajorkmsXiao ChenXiao Chen
HDFS-10206Datanodes not sorted properly by distance when the reader isn't a datanodeMajor.Ming MaNanda kumar
HDFS-11217Annotate NameNode and DataNode MXBean interfaces as Private/StableMajor.Akira AjisakaJagadesh Kiran N
YARN-4457Cleanup unchecked types for EventHandlerMajor.Daniel TempletonDaniel Templeton
HADOOP-13900Remove snapshot version of SDK dependency from Azure Data Lake Store File SystemMajorfs/adlVishwajeet DusaneVishwajeet Dusane
HDFS-10958Add instrumentation hooks around Datanode disk IOMajordatanodeXiaoyu YaoArpit Agarwal
HDFS-11249Redundant toString() in DFSConfigKeys.javaTrivial.Akira AjisakaJagadesh Kiran N
YARN-5882Test only changes from YARN-4126Major.Andrew WangJian He
HADOOP-13709Ability to clean up subprocesses spawned by Shell when the process exitsMajor.Eric BadgerEric Badger
HDFS-11262Remove unused variables in FSImage.javaTrivial.Akira AjisakaJagadesh Kiran N
HDFS-10930Refactor: Wrap Datanode IO related operationsMajordatanodeXiaoyu YaoXiaoyu Yao
HDFS-10959Adding per disk IO statistics and metrics in DataNode.MajordatanodeXiaoyu YaoXiaoyu Yao
HDFS-10913Introduce fault injectors to simulate slow mirrorsMajordatanode, testXiaobing ZhouXiaobing Zhou
HADOOP-13911Remove TRUSTSTORE_PASSWORD related scripts from KMSMinorkmsXiao ChenJohn Zhuge
YARN-4994Use MiniYARNCluster with try-with-resources in testsTrivialtestAndras BokorAndras Bokor
HADOOP-13863Azure: Add a new SAS key mode for WASB.Majorfs/azureDushyanthDushyanth
HDFS-10917Collect peer performance statistics on DataNode.MajordatanodeXiaobing ZhouXiaobing Zhou
YARN-5969FairShareComparator: Cache value of getResourceUsage for better performanceMajorfairschedulerzhangshilongzhangshilong
HDFS-11275Check groupEntryIndex and throw a helpful exception on failures when removing ACL.MajornamenodeXiao ChenXiao Chen
YARN-5709Cleanup leader election configs and pluggabilityCriticalresourcemanagerKarthik KambatlaKarthik Kambatla
HDFS-11279Cleanup unused DataNode#checkDiskErrorAsync()Minor.Xiaoyu YaoHanisha Koneru
HDFS-9483Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS.MajordocumentationChris NaurothSurendra Singh Lilhore
YARN-5991Yarn Distributed Shell does not print throwable t to App Master When failed to start containerMinor.dashwangJim Frankola
HDFS-11292log lastWrittenTxId etc info in logSyncAllMajorhdfsYongjun ZhangYongjun Zhang
HDFS-11273Move TransferFsImage#doGetUrl function to a Util classMajor.Hanisha KoneruHanisha Koneru
YARN-6015AsyncDispatcher thread name can be set to improved debuggingMajor.Ajith SAjith S
HADOOP-13953Make FTPFileSystem's data connection mode and transfer mode configurableMajorfsXiao ChenXiao Chen
HDFS-11299Support multiple Datanode File IO hooksMajorhdfsHanisha KoneruHanisha Koneru
YARN-5849Automatically create YARN control group for pre-mounted cgroupsMinor.Miklos SzegediMiklos Szegedi
HDFS-11339Support File IO sampling for Datanode IO profiling hooksMajordatanodeHanisha KoneruHanisha Koneru
HDFS-11274Datanode should only check the failed volume upon IO errorsMajordatanodeXiaoyu YaoXiaoyu Yao
MAPREDUCE-6819Replace UTF8 with Text in MRBenchMinortestAkira AjisakaPeter Bacsko
HDFS-11306Print remaining edit logs from buffer if edit log can't be rolled.Majorha, namenodeWei-Chiu ChuangWei-Chiu Chuang
HADOOP-13978Update project release notes for 3.0.0-alpha2MajordocumentationAndrew WangAndrew Wang
HADOOP-13955Replace deprecated HttpServer2 and SSLFactory constantsMinor.John ZhugeJohn Zhuge
HADOOP-13496Include file lengths in Mismatch in length error for distcpMinor.Ted YuTed Yu
HADOOP-13999Add -DskipShade maven profile to disable jar shading to reduce compile timeMinorbuildArun SureshArun Suresh
YARN-6028Add document for container metricsMajordocumentation, nodemanagerWeiwei YangWeiwei Yang
MAPREDUCE-6728Give fetchers hint when ShuffleHandler rejects a shuffling connectionMajormrv2Haibo ChenHaibo Chen
HADOOP-13606swift FS to add a service load metadata fileMajorfs/swiftSteve LoughranSteve Loughran
HADOOP-13037Refactor Azure Data Lake Store as an independent FileSystemMajorfs/adlShrikant NaiduVishwajeet Dusane
HDFS-11156Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST APIMajorwebhdfsWeiwei YangWeiwei Yang
HADOOP-13738DiskChecker should perform some disk IOMajor.Arpit AgarwalArpit Agarwal

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HDFS-9034"StorageTypeStats" Metric should not count failed storage.MajornamenodeArchana TSurendra Singh Lilhore
MAPREDUCE-4784TestRecovery occasionally failsMajormrv2, testJason LoweHaibo Chen
HDFS-10760DataXceiver#run() should not log InvalidToken exception as an errorMajor.Pan YuxuanPan Yuxuan
HDFS-10729Improve log message for edit loading failures caused by FS limit checks.MajornamenodeWei-Chiu ChuangWei-Chiu Chuang
HADOOP-13375o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flakyMajorsecurity, testMingliang LiuWeiwei Yang
HDFS-10820Reuse closeResponder to reset the response variable in DataStreamer#runMinor.Yiqun LinYiqun Lin
YARN-5555Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested.Minor.Eric PayneEric Payne
YARN-5549AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminatelyCriticalresourcemanagerDaniel TempletonDaniel Templeton
HDFS-10835Fix typos in httpfs.shTrivialhttpfsJohn ZhugeJohn Zhuge
HDFS-10841Remove duplicate or unused variable in appendFile()Minor.Kihwal LeeKihwal Lee
HADOOP-13558UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticketMajorsecurityAlejandro AbdelnurXiao Chen
HADOOP-13388Clean up TestLocalFileSystemPermissionTrivialfsAndras BokorAndras Bokor
HDFS-10844test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failingMajorlibhdfsAkira AjisakaAkira Ajisaka
HDFS-9038DFS reserved space is erroneously counted towards non-DFS used.MajordatanodeChris NaurothBrahma Reddy Battula
MAPREDUCE-6628Potential memory leak in CryptoOutputStreamMajorsecurityMariappan AsokanMariappan Asokan
HDFS-10832Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissionsCriticalhttpfsAndrew WangAndrew Wang
HDFS-9781FsDatasetImpl#getBlockReports can occasionally throw NullPointerExceptionMajordatanodeWei-Chiu ChuangManoj Govindassamy
HDFS-10830FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in useMajorhdfsManoj GovindassamyArpit Agarwal
HADOOP-13587distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't setMinortools/distcpZoran DimitrijevicZoran Dimitrijevic
HDFS-10856Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReportMinordocumentationAkira AjisakaYiqun Lin
YARN-5630NM fails to start after downgrade from 2.8 to 2.7BlockernodemanagerJason LoweJason Lowe
HADOOP-13616Broken code snippet area in Hadoop BenchmarkingMinordocumentationKai SasakiKai Sasaki
HDFS-10862Typos in 4 log messagesTrivial.Mehran HassaniMehran Hassani
YARN-5163Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engineMajortestArun SureshWei Zhou
YARN-4232TopCLI console support for HA modeMinor.Bibin A ChundattBibin A Chundatt
MAPREDUCE-6777Typos in 4 log messagesTrivial.Mehran HassaniMehran Hassani
YARN-5642Typos in 9 log messagesTrivial.Mehran HassaniMehran Hassani
YARN-5657Fix TestDefaultContainerExecutorMajortestAkira AjisakaArun Suresh
YARN-5577[Atsv2] Document object passing in infofilters with an exampleMajortimelinereader, timelineserverRohith Sharma K SRohith Sharma K S
YARN-5655TestContainerManagerSecurity#testNMTokens is assertingMajor.Jason LoweRobert Kanter
HADOOP-13601Fix typo in a log messages of AbstractDelegationTokenSecretManagerTrivial.Mehran HassaniMehran Hassani
HDFS-10879TestEncryptionZonesWithKMS#testReadWrite fails intermittentlyMajor.Xiao ChenXiao Chen
HADOOP-13602Fix some warnings by findbugs in hadoop-maven-pluginMajor.Tsuyoshi OzawaTsuyoshi Ozawa
YARN-4973YarnWebParams next.fresh.interval should be next.refresh.intervalMinorwebappDaniel TempletonDaniel Templeton
YARN-5539TimelineClient failed to retry on "java.net.SocketTimeoutException: Read timed out"CriticalyarnSumana SathishJunping Du
HADOOP-13643Math error in AbstractContractDistCpTestMinor.Aaron FabbriAaron Fabbri
HDFS-10894Remove the redundant charactors for command -saveNamespace in HDFSCommands.mdTrivialdocumentationYiqun LinYiqun Lin
HDFS-10886Replace "fs.default.name" with "fs.defaultFS" in viewfs documentMinordocumentation, federationBrahma Reddy BattulaBrahma Reddy Battula
HDFS-10843Update space quota when a UC block is completed rather than committed.Majorhdfs, namenodeErik KrogenErik Krogen
HDFS-10866Fix Eclipse Java 8 compile errors related to generic parameters.Major.Konstantin ShvachkoKonstantin Shvachko
YARN-5664Fix Yarn documentation to link to correct versions.Minor.Xiao ChenXiao Chen
YARN-5663Small refactor in ZKRMStateStoreMinorresourcemanagerOleksii DymytrovOleksii Dymytrov
HADOOP-13638KMS should set UGI's Configuration object properlyMajorkmsWei-Chiu ChuangWei-Chiu Chuang
HDFS-9885Correct the distcp counters name while displaying countersMinordistcpArchana TSurendra Singh Lilhore
YARN-5660Wrong audit constants are used in Get/Put of priority in RMWebServiceTrivialresourcemanagerRohith Sharma K SRohith Sharma K S
HDFS-10889Remove outdated Fault Injection Framework documentaionMajordocumentationBrahma Reddy BattulaBrahma Reddy Battula
HDFS-10713Throttle FsNameSystem lock warningsMajorlogging, namenodeArpit AgarwalHanisha Koneru
HDFS-10426TestPendingInvalidateBlock failed in trunkMajortestYiqun LinYiqun Lin
HDFS-10828Fix usage of FsDatasetImpl object lock in ReplicaMapBlocker.Arpit AgarwalArpit Agarwal
YARN-5631Missing refreshClusterMaxPriority usage in rmadmin help messageMinor.Kai SasakiKai Sasaki
HDFS-10376Enhance setOwner testingMajor.Yongjun ZhangJohn Zhuge
HDFS-10915Fix time measurement bug in TestDatanodeRestart#testWaitForRegistrationOnRestartMinortestXiaobing ZhouXiaobing Zhou
HDFS-9444Add utility to find set of available ephemeral ports to ServerSocketUtilMajor.Brahma Reddy BattulaMasatake Iwasaki
YARN-5662Provide an option to enable ContainerMonitorMajor.Jian HeJian He
HADOOP-11780Prevent IPC reader thread deathCriticalipcDaryn SharpDaryn Sharp
HDFS-10824MiniDFSCluster#storageCapacities has no effects on real capacityMajor.Xiaobing ZhouXiaobing Zhou
HDFS-10914Move remnants of oah.hdfs.client to hadoop-hdfs-clientCriticalhdfs-clientAndrew WangAndrew Wang
HADOOP-13164Optimize S3AFileSystem::deleteUnnecessaryFakeDirectoriesMinorfs/s3Rajesh BalamohanRajesh Balamohan
MAPREDUCE-6771RMContainerAllocator sends container diagnostics event after corresponding completion eventMajormrv2Haibo ChenHaibo Chen
HADOOP-13640Fix findbugs warning in VersionInfoMojo.javaMajor.Tsuyoshi OzawaYuanbo Liu
HDFS-10850getEZForPath should NOT throw FNFBlockerhdfsDaryn SharpAndrew Wang
HADOOP-13671Fix ClassFormatException in trunk build.MajorbuildKihwal LeeKihwal Lee
HDFS-10907Fix Erasure Coding documentationTrivialdocumentation, erasure-codingWei-Chiu ChuangManoj Govindassamy
YARN-5693Reduce loglevel to Debug in ContainerManagementProtocolProxy and AMRMClientImplMajoryarnYufei GuYufei Gu
YARN-5678Log demand as demand in FSLeafQueue and FSParentQueueMajorfairschedulerYufei GuYufei Gu
YARN-5672FairScheduler: wrong queue name in log when adding applicationMinorfairschedulerWilfred SpiegelenburgWilfred Spiegelenburg
YARN-4767Network issues can cause persistent RM UI outageCriticalwebappDaniel TempletonDaniel Templeton
HDFS-10810Setreplication removing block from underconstrcution temporarily when batch IBR is enabled.MajornamenodeBrahma Reddy BattulaBrahma Reddy Battula
HDFS-10944Correct the javadoc of dfsadmin#disallowSnapshotMinordocumentationJagadesh Kiran NJagadesh Kiran N
HDFS-10947Correct the API name for truncate in webhdfs documentMajordocumentationSurendra Singh LilhoreSurendra Singh Lilhore
HDFS-10878TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationExceptionMajorhdfs-clientRushabh S ShahRushabh S Shah
HADOOP-12667s3a: Support createNonRecursive APIMajorfs/s3Sean MackrorySean Mackrory
HDFS-10609Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applicationsMajorencryptionWei-Chiu ChuangWei-Chiu Chuang
HDFS-10962TestRequestHedgingProxyProvider is flakyMajortestAndrew WangAndrew Wang
MAPREDUCE-6789Fix TestAMWebApp failureMajortestAkira AjisakaDaniel Templeton
MAPREDUCE-6740Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.intervalMinormr-amHaibo ChenHaibo Chen
HADOOP-13690Fix typos in core-default.xmlMinordocumentationYiqun LinYiqun Lin
YARN-5101YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed orderMajor.Xuan GongSunil Govindan
YARN-5659getPathFromYarnURL should use standard methodsMajor.Sergey ShelukhinSergey Shelukhin
HADOOP-12611TestZKSignerSecretProvider#testMultipleInit occasionally failMajor.Wei-Chiu ChuangEric Badger
HDFS-10969Fix typos in hdfs-default.xmlMinordocumentationYiqun LinYiqun Lin
HDFS-10797Disk usage summary of snapshots causes renamed blocks to get counted twiceMajorsnapshotsSean MackrorySean Mackrory
YARN-5057resourcemanager.security.TestDelegationTokenRenewer fails in trunkMajor.Yongjun ZhangJason Lowe
HADOOP-13697LogLevel#main throws exception if no arguments providedMajor.Mingliang LiuMingliang Liu
HDFS-10991Export hdfsTruncateFile symbol in libhdfsBlockerlibhdfsSurendra Singh LilhoreSurendra Singh Lilhore
HADOOP-13700Remove unthrown IOException from TrashPolicy#initialize and #getInstance signaturesCriticalfsHaibo ChenAndrew Wang
HDFS-11002Fix broken attr/getfattr/setfattr links in ExtendedAttributes.mdMajordocumentationMingliang LiuMingliang Liu
HDFS-11000webhdfs PUT does not work if requests are routed to call queue.Major.Kihwal LeeKihwal Lee
HDFS-10987Make Decommission less expensive when lot of blocks present.Critical.Brahma Reddy BattulaBrahma Reddy Battula
HADOOP-13024Distcp with -delete feature on raw data not implementedMajortools/distcpMavin MartinMavin Martin
HDFS-10986DFSAdmin should log detailed error message if anyMajortoolsMingliang LiuMingliang Liu
HADOOP-13723AliyunOSSInputStream#read() should update read bytes stat correctlyMajortoolsMingliang LiuMingliang Liu
HDFS-10990TestPendingInvalidateBlock should wait for IBRsMinor.Yiqun LinYiqun Lin
HDFS-10735Distcp using webhdfs on secure HA clusters fails with StandbyExceptionMajorwebhdfsBenoy AntonyBenoy Antony
HDFS-10883`getTrashRoot`'s behavior is not consistent in DFS after enabling EZ.Major.Yuanbo LiuYuanbo Liu
HADOOP-13707If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessedMajor.Yuanbo LiuYuanbo Liu
HDFS-10301BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of orderCriticalnamenodeKonstantin ShvachkoVinitha Reddy Gankidi
HDFS-10920TestStorageMover#testNoSpaceDisk is failing intermittentlyMajortestRakesh RRakesh R
YARN-5743[Atsv2] Publish queue name and RMAppMetrics to ATSMajor.Rohith Sharma K SRohith Sharma K S
HDFS-10960TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume removeMinorhdfsManoj GovindassamyManoj Govindassamy
HDFS-10752Several log refactoring/improvement suggestion in HDFSMajor.Nemo ChenHanisha Koneru
HDFS-11025TestDiskspaceQuotaUpdate fails in trunk due to Bind exceptionMinor.Yiqun LinYiqun Lin
HDFS-10627Volume Scanner marks a block as "suspect" even if the exception is network-relatedMajorhdfsRushabh S ShahRushabh S Shah
HDFS-10699Log object instance get incorrectly in TestDFSAdminMinor.Yiqun LinYiqun Lin
HDFS-11018Incorrect check and message in FsDatasetImpl#invalidateMajordatanodeWei-Chiu ChuangYiqun Lin
HADOOP-13236truncate will fail when we use viewfilesystemMajor.Brahma Reddy BattulaBrahma Reddy Battula
HDFS-10730Fix some failed tests due to BindExceptionMajor.Yiqun LinYiqun Lin
YARN-5747Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishesMajortimelineserverLi LuLi Lu
YARN-5679TestAHSWebServices is failingMajortimelineserverAkira AjisakaAkira Ajisaka
HADOOP-13749KMSClientProvider combined with KeyProviderCache can result in wrong UGI being usedCritical.Sergey ShelukhinXiaoyu Yao
HDFS-11042Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()MajortestKuhu ShuklaKuhu Shukla
HADOOP-13626Remove distcp dependency on FileStatus serializationMajortools/distcpChris DouglasChris Douglas
HDFS-11046Duplicate '-' in the daemon log nameMajorloggingAkira AjisakaAkira Ajisaka
YARN-5711Propogate exceptions back to client when using hedging RM failover providerCriticalapplications, resourcemanagerSubru KrishnanSubru Krishnan
YARN-5754Null check missing for earliest in FifoPolicyMajorfairschedulerYufei GuYufei Gu
YARN-5777TestLogsCLI#testFetchApplictionLogsAsAnotherUser failsMajortestAkira AjisakaAkira Ajisaka
HDFS-11015Enforce timeout in balancerMajorbalancer & moverKihwal LeeKihwal Lee
HDFS-11040Add documentation for HDFS-9820 distcp improvementMajordistcpYongjun ZhangYongjun Zhang
YARN-5677RM should transition to standby when connection is lost for an extended periodCriticalresourcemanagerDaniel TempletonDaniel Templeton
HDFS-11054Suppress verbose log message in BlockPlacementPolicyDefaultMajor.Arpit AgarwalChen Liang
HDFS-10935TestFileChecksum fails in some casesMajor.Wei-Chiu ChuangSammi Chen
YARN-5753fix NPE in AMRMClientImpl.getMatchingRequests()MajoryarnHaibo ChenHaibo Chen
HDFS-11050Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATORMajor.Kuhu ShuklaKuhu Shukla
HDFS-11053Unnecessary superuser check in versionRequest()Majornamenode, securityKihwal LeeKihwal Lee
YARN-5433Audit dependencies for Category-XBlockertimelineserverSean BusbeySangjin Lee
HDFS-10921TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe modeMajor.Eric BadgerEric Badger
HADOOP-8299ViewFileSystem link slash mount point crashes with IndexOutOfBoundsExceptionMajorviewfsEli CollinsManoj Govindassamy
HDFS-9929Duplicate keys in NAMENODE_SPECIFIC_KEYSMinornamenodeAkira AjisakaAkira Ajisaka
YARN-5752TestLocalResourcesTrackerImpl#testLocalResourceCache times outMajor.Eric BadgerEric Badger
YARN-5710Fix inconsistent naming in class ResourceRequestTrivialyarnYufei GuYufei Gu
YARN-5686DefaultContainerExecutor random working dir algorigthm skews resultsMinor.Miklos SzegediVrushali C
HDFS-10769BlockIdManager.clear doesn't reset the counter for blockGroupIdGeneratorMinorhdfsEwan HiggsYiqun Lin
MAPREDUCE-6798Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods()MajorjobhistoryserverHaibo ChenHaibo Chen
HDFS-8492DN should notify NN when client requests a missing blockMajor.Daryn SharpWalter Su
YARN-5757RM REST API documentation is not up to dateTrivialresourcemanager, yarnMiklos SzegediMiklos Szegedi
MAPREDUCE-6541Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlockMajor.Wangda TanVarun Saxena
YARN-3848TestNodeLabelContainerAllocation is not timing outMajortestJason LoweVarun Saxena
YARN-5420Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessaryMinorresourcemanagerSunil GovindanSunil Govindan
HADOOP-13201Print the directory paths when ViewFs denies the rename operation on internal dirsMajorviewfsTianyin XuRakesh R
YARN-5172Update yarn daemonlog documentation due to HADOOP-12847TrivialdocumentationWei-Chiu ChuangWei-Chiu Chuang
YARN-4831Recovered containers will be killed after NM stateful restartMajornodemanagerSiqi LiSiqi Li
YARN-4388Cleanup "mapreduce.job.hdfs-servers" from yarn-default.xmlMinoryarnJunping DuJunping Du
YARN-5776Checkstyle: MonitoringThread.Run method length is too longTrivialnodemanagerMiklos SzegediMiklos Szegedi
YARN-3432Cluster metrics have wrong Total Memory when there is reserved memory on CSMajorcapacityscheduler, resourcemanagerThomas GravesBrahma Reddy Battula
HDFS-9500datanodesSoftwareVersions map may counting wrong when rolling upgradeMajor.Phil YangErik Krogen
MAPREDUCE-2631Potential resource leaks in BinaryProtocol$TeeOutputStream.javaMajor.Ravi Teja Ch N VSunil Govindan
YARN-2306Add test for leakage of reservation metrics in fair schedulerMinorfairschedulerHong ZhiguoHong Zhiguo
YARN-4743FairSharePolicy breaks TimSort assumptionMajorfairschedulerZephyr GuoZephyr Guo
HADOOP-13763KMS REST API Documentation Decrypt URL typoMinordocumentation, kmsJeffrey E RodriguezJeffrey E Rodriguez
YARN-5794Fix the asflicense warningsMajor.Yiqun LinYiqun Lin
HADOOP-13770Shell.checkIsBashSupported swallowed an interrupted exceptionMinorutilWei-Chiu ChuangWei-Chiu Chuang
YARN-5027NM should clean up app log dirs after NM restartMajornodemanagersandfleesandflee
YARN-5767Fix the order that resources are cleaned up from the local Public/Private cachesMajor.Chris TrezzoChris Trezzo
HDFS-11061Update dfs -count -t command line help and documentationMinordocumentation, fsWei-Chiu ChuangYiqun Lin
YARN-5773RM recovery too slow due to LeafQueue#activateApplication()Criticalcapacity scheduler, rolling upgradeBibin A ChundattBibin A Chundatt
YARN-5800Delete LinuxContainerExecutor comment from yarn-default.xmlTrivialyarnDaniel TempletonJan Hentschel
YARN-5809AsyncDispatcher possibly invokes multiple shutdown thread when handling exceptionMajor.Jian HeJian He
HADOOP-8500Fix javadoc jars to not contain entire target directoryMinorbuildEJ CiramellaAndrew Wang
HDFS-10455Logging the username when deny the setOwner operationMinornamenodeTianyin XuRakesh R
YARN-5805Add isDebugEnabled check for debug logs in nodemanagerMinor.Bibin A ChundattBibin A Chundatt
YARN-5788Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNodeMajor.Bibin A ChundattBibin A Chundatt
YARN-5001Aggregated Logs root directory is created with wrong group if nonexistentMajorlog-aggregation, nodemanager, securityHaibo ChenHaibo Chen
MAPREDUCE-6765MR should not schedule container requests in cases where reducer or mapper containers demand resource larger than the maximum supportedMinormr-amHaibo ChenHaibo Chen
YARN-5815Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestartMajor.Bibin A ChundattBibin A Chundatt
HDFS-11095BlockManagerSafeMode should respect extension period default config value (30s)MinornamenodeMingliang LiuMingliang Liu
YARN-4862Handle duplicate completed containers in RMNodeImplMajorresourcemanagerRohith Sharma K SRohith Sharma K S
HDFS-11097Fix the jenkins warning related to the deprecated method StorageReceivedDeletedBlocksMajor.Yiqun LinYiqun Lin
HADOOP-13787Azure testGlobStatusThrowsExceptionForUnreadableDir failsMinorfs/azureJohn ZhugeJohn Zhuge
HDFS-11098Datanode in tests cannot start in Windows after HDFS-10638BlockerdatanodeVinayakumar BVinayakumar B
YARN-5837NPE when getting node status of a decommissioned node after an RM restartMajor.Robert KanterRobert Kanter
HADOOP-13798TestHadoopArchives times outMajortestAkira AjisakaAkira Ajisaka
HADOOP-13797Remove hardcoded absolute path for lsMajorutilChristine KoppeltChristine Koppelt
HADOOP-13795Skip testGlobStatusThrowsExceptionForUnreadableDir in TestFSMainOperationsSwiftMajorfs/swift, testJohn ZhugeJohn Zhuge
HADOOP-13803Revert HADOOP-13081 add the ability to create multiple UGIs/subjects from one kerberos loginMajorsecurityAndrew WangChris Nauroth
YARN-5847Revert health check exit code checkMajornodemanagerAllen WittenauerAllen Wittenauer
HADOOP-13804MutableStat mean loses accuracy if add(long, long) is usedMinormetricsErik KrogenErik Krogen
YARN-5377Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainersMajor.Rohith Sharma K SKonstantinos Karanasos
MAPREDUCE-6782JHS task page search based on each individual column not workingMajorjobhistoryserverBibin A ChundattAjith S
HADOOP-13789Hadoop Common includes generated test protos in both jar and test-jarMajorbuild, commonSean BusbeySean Busbey
YARN-5823Update NMTokens in case of requests with only opportunistic containersBlocker.Konstantinos KaranasosKonstantinos Karanasos
HADOOP-13346DelegationTokenAuthenticationHandler writes via closed writerMinorsecurityGregory ChananHrishikesh Gadre
YARN-5856Unnecessary duplicate start container request sent to NM State storeMajor.Varun SaxenaVarun Saxena
YARN-5862TestDiskFailures.testLocalDirsFailures failedMajor.Yufei GuYufei Gu
YARN-5453FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResourceMajorfairschedulersandfleesandflee
YARN-5843Incorrect documentation for timeline service entityType/events REST end pointsMinor.Bibin A ChundattBibin A Chundatt
HDFS-9337Validate required params for WebHDFS requestsMajor.Jagadesh Kiran NJagadesh Kiran N
YARN-5834TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect valueTrivial.Miklos SzegediChang Li
HDFS-11128CreateEditsLog throws NullPointerExceptionMajorhdfsHanisha KoneruHanisha Koneru
YARN-5868Update npm to latest version in Dockerfile to avoid random failures of npm while run maven buildMajoryarn-ui-v2Sunil GovindanSunil Govindan
YARN-5545Fix issues related to Max App in capacity schedulerMajorcapacity schedulerBibin A ChundattBibin A Chundatt
YARN-5825ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized blockMajorcapacity schedulerSunil GovindanSunil Govindan
HDFS-11129TestAppendSnapshotTruncate fails with bind exceptionMajortestBrahma Reddy BattulaBrahma Reddy Battula
HADOOP-13813TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failingMajorsecurity, testMingliang LiuMingliang Liu
HDFS-11135The tests in TestBalancer run fails due to NPEMajortestYiqun LinYiqun Lin
HDFS-11056Concurrent append and read operations lead to checksum errorMajordatanode, httpfsWei-Chiu ChuangWei-Chiu Chuang
MAPREDUCE-6797Job history server scans can become blocked on a single, slow entryCriticaljobhistoryserverPrabhu JosephPrabhu Joseph
YARN-5874RM -format-state-store and -remove-application-from-state-store commands fail with NPECriticalresourcemanagerVarun SaxenaVarun Saxena
YARN-5873RM crashes with NPE if generic application history is enabledCriticalresourcemanagerVarun SaxenaVarun Saxena
YARN-4355NPE while processing localizer heartbeatMajornodemanagerJason LoweVarun Saxena
YARN-4218Metric for resource*time that was preemptedMajorresourcemanagerChang LiChang Li
YARN-5875TestTokenClientRMService#testTokenRenewalWrongUser failsMajor.Varun SaxenaGergely Novák
HADOOP-13815TestKMS#testDelegationTokensOpsSimple and TestKMS#testDelegationTokensOpsKerberized Fails in TrunkMajortestBrahma Reddy BattulaXiao Chen
YARN-5765Revert CHMOD on the new dirs created-LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.Blocker.Haibo ChenNaganarasimha G R
MAPREDUCE-6811TestPipeApplication#testSubmitter fails after HADOOP-13802MajortestBrahma Reddy BattulaBrahma Reddy Battula
YARN-5891yarn rmadmin -help contains a misspelled ResourceManagerTrivialresourcemanagerGrant SohnGrant Sohn
YARN-5836Malicious AM can kill containers of other apps running in any node its containers are runningMinornodemanagerBotong HuangBotong Huang
YARN-5870Expose getApplications API in YarnClient with GetApplicationsRequest parameterMajorclientGour SahaJian He
HDFS-11134Fix bind exception threw in TestRenameWhileOpenMajor.Yiqun LinYiqun Lin
YARN-3538TimelineServer doesn't catch/translate all exceptions raisedMinortimelineserverSteve LoughranSteve Loughran
YARN-5904Reduce the number of default server threads for AMRMProxyServiceMinornodemanagerSubru KrishnanSubru Krishnan
MAPREDUCE-6801Fix flaky TestKill.testKillJob()Majormrv2Haibo ChenHaibo Chen
HADOOP-13814Sample configuration of KMS HTTP Authentication signature is misleadingMinorconf, documentation, kmsMasahiro TanakaMasahiro Tanaka
HDFS-11144TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exceptionMajortestBrahma Reddy BattulaBrahma Reddy Battula
HDFS-11101TestDFSShell#testMoveWithTargetPortEmpty fails intermittentlyMajortestBrahma Reddy BattulaBrahma Reddy Battula
YARN-5859TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes failsMajortestJason LoweEric Badger
MAPREDUCE-6793io.sort.factor code default and mapred-default.xml values inconsistentTrivialtaskGera ShegalovPrabhu Joseph
HDFS-10966Enhance Dispatcher logic on deciding when to give up a source DataNodeMajorbalancer & moverZhe ZhangMark Wagner
HADOOP-13663Index out of range in SysInfoWindowsMajornative, utilÍñigo GoiriÍñigo Goiri
YARN-5911DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is trueMajor.Varun SaxenaVarun Saxena
YARN-5918Handle Opportunistic scheduling allocate request failure when NM is lostMajor.Bibin A ChundattBibin A Chundatt
HADOOP-13766Fix a typo in the comments of RPC.getProtocolVersionTrivialcommonEthan LiEthan Li
YARN-5920Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHangMajortestRohith Sharma K SVarun Saxena
HADOOP-13833TestSymlinkHdfsFileSystem#testCreateLinkUsingPartQualPath2 fails after HADOOP13605CriticaltestBrahma Reddy BattulaBrahma Reddy Battula
HDFS-11174Wrong HttpFS test command in docMinordocumentation, httpfsJohn ZhugeJohn Zhuge
HADOOP-13820Replace ugi.getUsername() with ugi.getShortUserName() in viewFSMinorviewfsArchana TBrahma Reddy Battula
YARN-5572HBaseTimelineWriterImpl appears to reference a bad property nameMajor.Daniel TempletonVarun Saxena
HADOOP-13816Ambiguous plugin version warning from maven build.Minor.Kai SasakiKai Sasaki
HADOOP-13838KMSTokenRenewer should close providersCriticalkmsXiao ChenXiao Chen
YARN-5725Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange when setting IP and hostMinornodemanagerMiklos SzegediMiklos Szegedi
YARN-5851TestContainerManagerSecurity testContainerManager[1] failedMajornodemanagerHaibo ChenHaibo Chen
MAPREDUCE-6565Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution.Major.Chris NaurothLi Lu
YARN-5942"Overridden" is misspelled as "overriden" in FairScheduler.mdTrivialsiteDaniel TempletonHeather Sutherland
HADOOP-13830Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string"Minorfs/s3, testSteve LoughranSteve Loughran
YARN-5915ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event writeMajortimelineserverAtul SikariaAtul Sikaria
MAPREDUCE-6815Fix flaky TestKill.testKillTask()Majormrv2Haibo ChenHaibo Chen
YARN-5901Fix race condition in TestGetGroups beforeclass setup()MajoryarnHaibo ChenHaibo Chen
YARN-5929Missing scheduling policy in the FS queue metric.Major.Yufei GuYufei Gu
HDFS-11181Fuse wrapper has a typoTrivialfuse-dfsWei-Chiu ChuangWei-Chiu Chuang
HADOOP-13847KMSWebApp should close KeyProviderCryptoExtensionMajorkmsAnthony Young-GarnerJohn Zhuge
YARN-5559Analyse 2.8.0/3.0.0 jdiff reports and fix any issuesBlockerresourcemanagerWangda TanAkira Ajisaka
HADOOP-13675Bug in return value for delete() calls in WASBMajorfs/azureDushyanthDushyanth
HADOOP-13864KMS should not require truststore passwordMajorkms, securityMike YoderMike Yoder
HADOOP-13861Spelling errors in logging and exceptions for codeMajorcommon, fs, io, securityGrant SohnGrant Sohn
HDFS-11180Intermittent deadlock in NameNode when failover happens.BlockernamenodeAbhishek ModiAkira Ajisaka
HDFS-11198NN UI should link DN web address using hostnamesCritical.Kihwal LeeWeiwei Yang
MAPREDUCE-6571JobEndNotification info logs are missing in AM container syslogMinorapplicationmasterPrabhu JosephHaibo Chen
MAPREDUCE-6816Progress bars in Web UI always at 100%BlockerwebappsShen YinjieShen Yinjie
HADOOP-13859TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties.MajorcommonHaibo ChenHaibo Chen
YARN-5184Fix up incompatible changes introduced on ContainerStatus and NodeReportBlockerapiKarthik KambatlaSangjin Lee
YARN-5932Retrospect moveApplicationToQueue in align with YARN-5611Majorcapacity scheduler, resourcemanagerSunil GovindanSunil Govindan
YARN-5136Error in handling event type APP_ATTEMPT_REMOVED to the schedulerMajor.tangshangwenWilfred Spiegelenburg
MAPREDUCE-6817The format of job start time in JHS is different from those of submit and finish timeMajorjobhistoryserverHaibo ChenHaibo Chen
YARN-5963Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history codeTrivialclient, nodemanagerGrant SohnGrant Sohn
YARN-5921Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceStateMajor.Varun SaxenaVarun Saxena
HDFS-11140Directory Scanner should log startup message time correctlyMinor.Xiao ChenYiqun Lin
HDFS-11223Fix typos in HttpFs documentationsTrivialdocumentationYiqun LinYiqun Lin
HADOOP-13867FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementationsMajor.Vinayakumar BVinayakumar B
HDFS-11197Listing encryption zones fails when deleting a EZ that is on a snapshotted directoryMinorhdfsWellington ChevreuilWellington Chevreuil
HDFS-11224Lifeline message should be ignored for dead nodesCritical.Vinayakumar BVinayakumar B
YARN-4752FairScheduler should preempt for a ResourceRequest and all preempted containers should be on the same nodeMajorfairschedulerKarthik KambatlaKarthik Kambatla
HADOOP-13824FsShell can suppress the real error if no error message is presentMajorfsRob VesseJohn Zhuge
MAPREDUCE-6820Fix dead links in Job relevant classesMinordocumentationYiqun LinYiqun Lin
HDFS-8870Lease is leaked on write failureMajorhdfs-clientRushabh S ShahKuhu Shukla
HADOOP-13565KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client requestMajorsecurityXiaoyu YaoXiaoyu Yao
HDFS-11229HDFS-11056 failed to close meta fileBlockerdatanodeWei-Chiu ChuangWei-Chiu Chuang
HDFS-11226cacheadmin,cryptoadmin and storagepolicyadmin should support generic optionsMinortoolsArchana TBrahma Reddy Battula
HDFS-11233Fix javac warnings related to the deprecated APIs after upgrading JacksonMinor.Yiqun LinYiqun Lin
MAPREDUCE-6821Fix javac warning related to the deprecated APIs after upgrading JacksonMinor.Yiqun LinYiqun Lin
HDFS-11164Mover should avoid unnecessary retries if the block is pinnedMajorbalancer & moverRakesh RRakesh R
HDFS-10684WebHDFS DataNode calls fail without parameter createparentBlockerwebhdfsSamuel LowJohn Zhuge
HDFS-11204Document the missing options of hdfs zkfc commandMinordocumentationYiqun LinYiqun Lin
HADOOP-13890Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsingMajortestBrahma Reddy BattulaXiaoyu Yao
YARN-5999AMRMClientAsync will stop if any exceptions thrown on allocate callMajor.Jian HeJian He
HADOOP-13831Correct check for error code to detect Azure Storage Throttling and provide retriesMajorfs/azureGaurav KanadeGaurav Kanade
HADOOP-13508FsPermission string constructor does not recognize sticky bitMajor.Atul SikariaAtul Sikaria
HDFS-11094Send back HAState along with NamespaceInfo during a versionRequest as an optional parameterMajordatanodeEric BadgerEric Badger
HDFS-11253FileInputStream leak on failure path in BlockSenderMajordatanodeArpit AgarwalArpit Agarwal
HDFS-9911TestDataNodeLifeline Fails intermittentlyMajordatanodeAnu EngineerYiqun Lin
HDFS-11160VolumeScanner reports write-in-progress replicas as corrupt incorrectlyMajordatanodeWei-Chiu ChuangWei-Chiu Chuang
HADOOP-13913maven issuing pom.version being deprecation warningsMinorbuildJoe PallasSteve Loughran
YARN-4330MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages.Blockertest, yarnSteve LoughranVarun Saxena
YARN-5877Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launchMajor.Bibin A ChundattBibin A Chundatt
HDFS-11263ClassCastException when we use Bzipcodec for Fsimage compressionCritical.Brahma Reddy BattulaBrahma Reddy Battula
HDFS-11195Return error when appending files by webhdfs rest api failsMajor.Yuanbo LiuYuanbo Liu
HDFS-11247Add a test to verify NameNodeMXBean#getDecomNodes() and Live/Dead Decom Nodes shown in NameNode WebUIMajor.Manoj GovindassamyManoj Govindassamy
HDFS-11261Document missing NameNode metricsMinordocumentationYiqun LinYiqun Lin
YARN-5903Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup methodMajoryarnHaibo ChenHaibo Chen
YARN-5774MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0.BlockerresourcemanagerYufei GuYufei Gu
HDFS-11258File mtime change could not save to editlogCritical.Jimmy XiangJimmy Xiang
MAPREDUCE-6704Update the documents to run MapReduce applicationBlockerdocumentationBibin A ChundattBibin A Chundatt
YARN-6000Make AllocationFileLoaderService.Listener publicMajorfairscheduler, yarnTao JieTao Jie
HDFS-11271Typo in NameNode UITrivialnamenode, uiWei-Chiu ChuangWei-Chiu Chuang
HDFS-11250Fix a typo in ReplicaUnderRecovery#setRecoveryIDTrivial.Yiqun LinYiqun Lin
YARN-6026A couple of spelling errors in the docsTrivialdocumentationGrant SohnGrant Sohn
HADOOP-13940Document the missing envvars commandsMinordocumentationYiqun LinYiqun Lin
HADOOP-13943TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863MajortestBrahma Reddy BattulaBrahma Reddy Battula
HDFS-11270Document the missing options of NameNode bootstrap commandMinordocumentationYiqun LinYiqun Lin
HADOOP-13932Fix indefinite article in commentsMinor.LiXin GeLiXin Ge
YARN-5962Spelling errors in logging and exceptions for resource manager codeTrivialresourcemanagerGrant SohnGrant Sohn
YARN-5257Fix unreleased resources and null dereferencesMajor.Yufei GuYufei Gu
HDFS-11252TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindExceptionMajor.Jason LoweYiqun Lin
YARN-6001Improve moveApplicationQueues command lineMajorclientSunil GovindanSunil Govindan
YARN-6024Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to maxMajor.Wangda TanWangda Tan
YARN-4882Change the log level to DEBUG for recovering completed applicationsMajorresourcemanagerRohith Sharma K SDaniel Templeton
HDFS-11251ConcurrentModificationException during DataNode#refreshVolumesMajor.Jason LoweManoj Govindassamy
HDFS-11267Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in StorageMajor.Manoj GovindassamyManoj Govindassamy
MAPREDUCE-5155Race condition in test case TestFetchFailure cause it to failMinortestNemon LouHaibo Chen
HADOOP-13942Build failure due to errors of javadoc build in hadoop-azureMajorfs/azureKai SasakiKai Sasaki
HADOOP-13883Add description of -fs option in generic command usageMinordocumentationYiqun LinYiqun Lin
HADOOP-13922Some modules have dependencies on hadoop-client jar removed by HADOOP-11804BlockerbuildJoe PallasSean Busbey
HADOOP-12733Remove references to obsolete io.seqfile configuration variablesMinor.Ray ChiangRay Chiang
YARN-5988RM unable to start in secure setupBlocker.Ajith SAjith S
HDFS-11280Allow WebHDFS to reuse HTTP connections to NNMajorhdfsZheng ShaoZheng Shao
HADOOP-13896Invoke assembly plugin after building jarsBlockerbuildAllen WittenauerAndrew Wang
HADOOP-13780LICENSE/NOTICE are out of date for source artifactsBlockercommonSean BusbeyXiao Chen
MAPREDUCE-6715Fix Several Unsafe PracticesMajor.Yufei GuYufei Gu
HDFS-11282Document the missing metrics of DataNode Volume IO operationsMinordocumentationYiqun LinYiqun Lin
MAPREDUCE-6711JobImpl fails to handle preemption events on state COMMITTINGMajor.Li LuPrabhu Joseph
YARN-6066Opportunistic containers minor fixes: API annotations and config parameter changesMinor.Arun SureshArun Suresh
HADOOP-13958Bump up release year to 2017Blocker.Junping DuJunping Du
YARN-6068Log aggregation get failed when NM restart even with recoveryBlocker.Junping DuJunping Du
HDFS-11301Double wrapping over RandomAccessFile in LocalReplicaInPipeline#createStreamsMinorhdfsHanisha KoneruHanisha Koneru
YARN-6073Misuse of format specifier in Preconditions.checkArgumentTrivial.Yongjun ZhangYuanbo Liu
YARN-5899Debug log in AbstractCSQueue#canAssignToThisQueue needs improvementTrivialcapacity schedulerYing ZhangYing Zhang
YARN-6054TimelineServer fails to start when some LevelDb state files are missing.Critical.Ravi PrakashRavi Prakash
YARN-5937stop-yarn.sh is not able to gracefully stop node managersMajor.Weiwei YangWeiwei Yang
YARN-6022Revert changes of AbstractResourceRequestBlocker.Wangda TanWangda Tan
YARN-4148When killing app, RM releases app's resource before they are released by NMMajorresourcemanagerJun GongJason Lowe
YARN-6079simple spelling errors in yarn test codeTrivialtestGrant Sohnvijay
HADOOP-13903Improvements to KMS logging to help debug authorization errorsMinorkmsTristan StevensTristan Stevens
HDFS-9935Remove LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD and unused import from HdfsServerConstantsMinor.Yiqun LinYiqun Lin
YARN-5554MoveApplicationAcrossQueues does not check user permission on the target queueMajorresourcemanagerHaibo ChenWilfred Spiegelenburg
HDFS-11312Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProtoBlocker.Sean MackrorySean Mackrory
HADOOP-13961Fix compilation failure from missing hadoop-kms test jarBlockerbuildKarthik KambatlaAndrew Wang
YARN-6072RM unable to start in secure modeBlockerresourcemanagerBibin A ChundattAjith S
YARN-6081LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved containerCritical.Wangda TanWangda Tan
HDFS-11344The default value of the setting dfs.disk.balancer.block.tolerance.percent is differentTrivialdiskbalancerYiqun LinYiqun Lin
HADOOP-13928TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime errorMajorfs/adl, testJohn ZhugeJohn Zhuge
HDFS-11342Fix FileInputStream leak in loadLastPartialChunkChecksumMajordatanodeArpit AgarwalChen Liang
HDFS-11307The rpc to portmap service for NFS has hardcoded timeout.Major.Jitendra Nath PandeyMukul Kumar Singh
YARN-6057yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of boundsMinor.Bibin A ChundattJulia Sommer
HADOOP-13976Path globbing does not match newlinesMajor.Eric BadgerEric Badger
HDFS-11209SNN can't checkpoint when rolling upgrade is not finalizedCriticalrolling upgradesXiaoyu YaoXiaoyu Yao
HDFS-10733NameNode terminated after full GC thinking QJM is unresponsive.Majornamenode, qjmKonstantin ShvachkoVinitha Reddy Gankidi
HDFS-10759Change fsimage bool isStriped from boolean to an enumMajorhdfsEwan HiggsEwan Higgs
HADOOP-13965Groups should be consistent in using default group mapping classMinorsecurityYiqun LinYiqun Lin
HDFS-11316TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunkMinor.Yiqun LinYiqun Lin
HDFS-11287Storage class member storageDirs should be private to avoid unprotected access by derived classesMajor.Manoj GovindassamyManoj Govindassamy
HDFS-11268Correctly reconstruct erasure coding file from FSImageCriticalerasure-codingSammi ChenSammi Chen
HADOOP-14001Improve delegation token validity checkingMajor.Akira AjisakaAkira Ajisaka
YARN-6110Fix opportunistic containers documentationMinordocumentationAkira AjisakaAkira Ajisaka
YARN-6104RegistrySecurity overrides zookeeper sasl system propertiesMajor.Billie RinaldiBillie Rinaldi
HADOOP-14004Missing hadoop-cloud-storage-project module in pom.xmlCriticalbuildAkira AjisakaAkira Ajisaka
YARN-5271ATS client doesn't work with Jersey 2 on the classpathMajorclient, timelineserverSteve LoughranWeiwei Yang
HDFS-11132Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing filesMajorfs/adl, testVishwajeet DusaneVishwajeet Dusane
HADOOP-13996Fix some release build issuesBlockerbuildAndrew WangAndrew Wang

TESTS:

JIRASummaryPriorityComponentReporterContributor
HDFS-10657testAclCLI.xml setfacl test should expect mask r-xMinor.John ZhugeJohn Zhuge
YARN-5656Fix ReservationACLsTestBaseMajor.Sean PoSean Po
HDFS-9333Some tests using MiniDFSCluster errored complaining port in useMinortestKai ZhengMasatake Iwasaki
HADOOP-13686Adding additional unit test for Trash (I)Major.Xiaoyu YaoWeiwei Yang
YARN-4555TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environmentMinornodemanager, testTakashi OhnishiTakashi Ohnishi
YARN-4363In TestFairScheduler, testcase should not create FairScheduler redundantlyTrivialfairschedulerTao JieTao Jie
YARN-3568TestAMRMTokens should use some random portMajortestGera ShegalovTakashi Ohnishi
YARN-3460TestSecureRMRegistryOperations fails with IBM_JAVAMajor.pascal olivapascal oliva
MAPREDUCE-6804Add timeout when starting JobHistoryServer in MiniMRYarnClusterMinortestAndras BokorAndras Bokor
MAPREDUCE-6743nativetask unit tests need to provide usable output; fix link errors during mvn testMajornativetaskAllen WittenauerAllen Wittenauer
HDFS-11272Refine the assert messages in TestFSDirAttrOpMinortestAkira AjisakaJimmy Xiang
HDFS-11278Add missing @Test annotation for TestSafeMode.testSafeModeUtils()TrivialnamenodeLukas MajercakLukas Majercak
MAPREDUCE-6831Flaky test TestJobImpl.testKilledDuringKillAbortMajormrv2Peter BacskoPeter Bacsko
HDFS-11290TestFSNameSystemMBean should wait until JMX cache is clearedMajortestAkira AjisakaErik Krogen
YARN-5608TestAMRMClient.setup() fails with ArrayOutOfBoundsExceptionMajortestDaniel TempletonDaniel Templeton

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor
YARN-4733[YARN-3368] Initial commit of new YARN web UIMajor.Wangda TanWangda Tan
YARN-4517[YARN-3368] Add nodes pageMajoryarnWangda TanVarun Saxena
YARN-4514[YARN-3368] Cleanup hardcoded configurations, such as RM/ATS addressesMajor.Wangda TanSunil Govindan
YARN-5019[YARN-3368] Change urls in new YARN ui from camel casing to hyphensMajor.Varun VasudevSunil Govindan
YARN-5000[YARN-3368] App attempt page is not loading when timeline server is not startedMajor.Sunil GovindanSunil Govindan
YARN-5038[YARN-3368] Application and Container pages shows wrong values when RM is stoppedMajor.Sunil GovindanSunil Govindan
YARN-4515[YARN-3368] Support hosting web UI framework inside YARN RMMajor.Wangda TanSunil Govindan
YARN-5183[YARN-3368] Support for responsive navbar when window is resizedMajor.Kai SasakiKai Sasaki
YARN-5161[YARN-3368] Add Apache Hadoop logo in YarnUI home pageMajorwebappSunil GovindanKai Sasaki
YARN-5344[YARN-3368] Generic UI improvementsMajor.Sreenath SomarajapuramSreenath Somarajapuram
YARN-5345[YARN-3368] Cluster overview page improvementsMajor.Sreenath SomarajapuramSreenath Somarajapuram
YARN-5346[YARN-3368] Queues page improvementsMajor.Sreenath SomarajapuramSreenath Somarajapuram
YARN-5347[YARN-3368] Applications page improvementsMajor.Sreenath SomarajapuramSreenath Somarajapuram
YARN-5348[YARN-3368] Node details page improvementsMajor.Sreenath SomarajapuramSreenath Somarajapuram
YARN-5321[YARN-3368] Add resource usage for application by node managersMajor.Wangda TanWangda Tan
YARN-5322[YARN-3368] Add a node heat chart mapMajor.Wangda TanWangda Tan
YARN-5334[YARN-3368] Introduce REFRESH button in various UI pagesMajorwebappSunil GovindanSreenath Somarajapuram
YARN-5509Build error due to preparing 3.0.0-alpha2 deploymentMajoryarnKai SasakiKai Sasaki
YARN-5488Applications table overflows beyond the page boundaryMajor.Harish JaiprakashHarish Jaiprakash
YARN-5504[YARN-3368] Fix YARN UI build pom.xmlMajor.Sreenath SomarajapuramSreenath Somarajapuram
HADOOP-13355Handle HADOOP_CLIENT_OPTS in a functionMajorscriptsAllen WittenauerAllen Wittenauer
YARN-5583[YARN-3368] Fix wrong paths in .gitignoreMajor.Sreenath SomarajapuramSreenath Somarajapuram
HADOOP-13554Add an equivalent of hadoop_subcmd_opts for secure optsMajorscriptsAllen WittenauerAllen Wittenauer
YARN-5503[YARN-3368] Add missing hidden files in webapp folder for deploymentMajor.Sreenath SomarajapuramSreenath Somarajapuram
HADOOP-13562Change hadoop_subcommand_opts to use only uppercaseMajorscriptsAllen WittenauerAllen Wittenauer
HADOOP-13358Modify HDFS to use hadoop_subcommand_optsMajorscriptsAllen WittenauerAllen Wittenauer
HADOOP-13356Add a function to handle command_subcommand_OPTSMajorscriptsAllen WittenauerAllen Wittenauer
HADOOP-13357Modify common to use hadoop_subcommand_optsMajorscriptsAllen WittenauerAllen Wittenauer
HADOOP-13359Modify YARN to use hadoop_subcommand_optsMajorscriptsAllen WittenauerAllen Wittenauer
HDFS-9392Admins support for maintenance stateMajor.Ming MaMing Ma
HADOOP-13564modify mapred to use hadoop_subcommand_optsMajorscriptsAllen WittenauerAllen Wittenauer
HDFS-10813DiskBalancer: Add the getNodeList method in CommandMinorbalancer & moverYiqun LinYiqun Lin
HADOOP-13563hadoop_subcommand_opts should print name not actual content during debugMajorscriptsAllen WittenauerAllen Wittenauer
HADOOP-13360Documentation for HADOOP_subcommand_OPTSMajorscriptsAllen WittenauerAllen Wittenauer
YARN-5221Expose UpdateResourceRequest API to allow AM to request for change in container propertiesMajor.Arun SureshArun Suresh
YARN-5596Fix failing unit test in TestDockerContainerRuntimeMinornodemanager, yarnSidharta SeethanaSidharta Seethana
HADOOP-13547Optimize IPC client protobuf decodingMajor.Daryn SharpDaryn Sharp
YARN-5264Store all queue-specific information in FSQueueMajorfairschedulerYufei GuYufei Gu
YARN-5576Allow resource localization while container is runningMajor.Jian HeJian He
HADOOP-13549Eliminate intermediate buffer for server-side PB encodingMajoripcDaryn SharpDaryn Sharp
HADOOP-13447Refactor S3AFileSystem to support introduction of separate metadata repository and tests.Majorfs/s3Chris NaurothChris Nauroth
YARN-5598[YARN-3368] Fix create-release to be able to generate bits for the new yarn-uiMajoryarn, yarn-ui-v2Wangda TanWangda Tan
HDFS-9847HDFS configuration should accept time unitsMajor.Yiqun LinYiqun Lin
HADOOP-13541explicitly declare the Joda time version S3A depends onMinorbuild, fs/s3Steve LoughranSteve Loughran
HDFS-8901Use ByteBuffer in striping positional readMajorerasure-codingKai ZhengSammi Chen
HDFS-10845Change defaults in hdfs-site.xml to match timeunit typeMinordatanode, namenodeYiqun LinYiqun Lin
HDFS-10553DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLIMinorbalancer & moverAnu EngineerManoj Govindassamy
HDFS-9849DiskBalancer : reduce lock path in shutdown codeMajorbalancer & moverAnu EngineerYuanbo Liu
YARN-5566Client-side NM graceful decom is not triggered when jobs finishMajornodemanagerRobert KanterRobert Kanter
HADOOP-10940RPC client does no bounds checking of responsesCriticalipcDaryn SharpDaryn Sharp
HADOOP-13540improve section on troubleshooting s3a auth problemsMinordocumentation, fs/s3Steve LoughranSteve Loughran
HDFS-10808DiskBalancer does not execute multi-steps plan-reduxMajorbalancer & moverAnu EngineerAnu Engineer
HDFS-10821DiskBalancer: Report command support with multiple nodesMajorbalancer & moverYiqun LinYiqun Lin
HDFS-10858FBR processing may generate incorrect reportedBlock-blockGroup mappingBlockererasure-codingJing ZhaoJing Zhao
HDFS-10599DiskBalancer: Execute CLI via ShellMajorbalancer & moverAnu EngineerManoj Govindassamy
HADOOP-13546Override equals and hashCode to avoid connection leakageMajoripcXiaobing ZhouXiaobing Zhou
HDFS-10562DiskBalancer: update documentation on how to report issues and debugMinorbalancer & moverAnu EngineerAnu Engineer
HDFS-10805Reduce runtime for append testMinortestGergely NovákGergely Novák
YARN-5620Core changes in NodeManager to support re-initialization of Containers with new launchContextMajor.Arun SureshArun Suresh
HADOOP-13621s3:// should have been fully cut off from trunkMajordocumentation, fs/s3Mingliang LiuMingliang Liu
YARN-5637Changes in NodeManager to support Container rollback and commitMajor.Arun SureshArun Suresh
YARN-3140Improve locks in AbstractCSQueue/LeafQueue/ParentQueueMajorresourcemanager, schedulerWangda TanWangda Tan
YARN-3141Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerAppMajorresourcemanager, schedulerWangda TanWangda Tan
HDFS-10871DiskBalancerWorkItem should not import jackson relocated by htraceMajorhdfs-clientMasatake IwasakiManoj Govindassamy
YARN-5609Expose upgrade and restart API in ContainerManagementProtocolMajor.Arun SureshArun Suresh
HDFS-9895Remove unnecessary conf cache from DataNodeMajordatanodeXiaobing ZhouXiaobing Zhou
HADOOP-13544JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIsBlocker.Vinod Kumar VavilapalliVinod Kumar Vavilapalli
YARN-3142Improve locks in AppSchedulingInfoMajorresourcemanager, schedulerWangda TanVarun Saxena
HDFS-10900DiskBalancer: Complete the documents for the report commandMinordocumentationYiqun LinYiqun Lin
HDFS-9850DiskBalancer : Explore removing references to FsVolumeSpiMajorbalancer & moverAnu EngineerManoj Govindassamy
HDFS-10779Rename does not need to re-solve destinationMajorhdfsDaryn SharpDaryn Sharp
HDFS-10892Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'Majorfs, shell, testMingliang LiuMingliang Liu
HADOOP-13599s3a close() to be non-synchronized, so avoid risk of deadlock on shutdownMajorfs/s3Steve LoughranSteve Loughran
YARN-4205Add a service for monitoring application life time outMajorschedulernijelRohith Sharma K S
YARN-5486Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requestsMajorresourcemanagerArun SureshKonstantinos Karanasos
HADOOP-12974Create a CachingGetSpaceUsed implementation that uses dfMajor.Elliott ClarkElliott Clark
HDFS-10851FSDirStatAndListingOp: stop passing path as stringMajorhdfsDaryn SharpDaryn Sharp
YARN-5384Expose priority in ReservationSystem submission APIsMajorcapacity scheduler, fairscheduler, resourcemanagerSean PoSean Po
HDFS-10619Cache path in InodesInPathMajorhdfsDaryn SharpDaryn Sharp
HDFS-10934TestDFSShell.testStat fails intermittentlyMajortestEric BadgerEric Badger
YARN-5682[YARN-3368] Fix maven build to keep all generated or downloaded files in target folderMajor.Wangda TanWangda Tan
HADOOP-13674S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.Minorfs/s3Chris NaurothChris Nauroth
HDFS-10956Remove rename/delete performance penalty when not using snapshotsMajorhdfsDaryn SharpDaryn Sharp
HDFS-10896Move lock logging logic from FSNamesystem into FSNamesystemLockMajornamenodeErik KrogenErik Krogen
YARN-5702Refactor TestPBImplRecords so that we can reuse for testing protocol records in other YARN modulesMajor.Subru KrishnanSubru Krishnan
YARN-3139Improve locks in AbstractYarnScheduler/CapacityScheduler/FairSchedulerMajorresourcemanager, schedulerWangda TanWangda Tan
HDFS-10893Refactor TestDFSShell by setting up MiniDFSCluser once for all commands testMajortestMingliang LiuMingliang Liu
HDFS-10955Pass IIP for FSDirAttr methodsMajorhdfsDaryn SharpDaryn Sharp
HDFS-10826Correctly report missing EC blocks in FSCKMajorerasure-codingTakanobu AsanumaTakanobu Asanuma
HADOOP-12977s3a to handle delete("/", true) robustlyMinorfs/s3Steve LoughranSteve Loughran
HDFS-10939Reduce performance penalty of encryption zonesMajorhdfsDaryn SharpDaryn Sharp
HADOOP-13692hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.Minorfs/s3Chris NaurothChris Nauroth
HDFS-10979Pass IIP for FSDirDeleteOp methodsMajorhdfsDaryn SharpDaryn Sharp
HADOOP-13627Have an explicit KerberosAuthException for UGI to throw, text from public constantsMajorsecuritySteve LoughranXiao Chen
HDFS-10980Optimize check for existence of parent directoryMajorhdfsDaryn SharpDaryn Sharp
HDFS-10968BlockManager#isInNewRack should consider decommissioning nodesMajorerasure-coding, namenodeJing ZhaoJing Zhao
HDFS-10988Refactor TestBalancerBandwidthMajorbalancer & mover, testBrahma Reddy BattulaBrahma Reddy Battula
HDFS-10985o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertionsMinorha, testMingliang LiuMingliang Liu
HDFS-10972Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'Majorfs, shell, testXiaobing ZhouXiaobing Zhou
HDFS-10965Add unit test for HDFS command 'dfsadmin -printTopology'Majorfs, shell, testXiaobing ZhouXiaobing Zhou
YARN-5698[YARN-3368] Launch new YARN UI under hadoop web app portMajor.Sunil GovindanSunil Govindan
HDFS-10949DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacityMinorbalancer & moverXiaobing ZhouXiaobing Zhou
HADOOP-13417Fix javac and checkstyle warnings in hadoop-auth packageMajor.Kai SasakiKai Sasaki
HDFS-10827When there are unrecoverable ec block groups, Namenode Web UI shows "There are X missing blocks." but doesn't show the block names.Majorerasure-codingTakanobu AsanumaTakanobu Asanuma
HDFS-11008Change unit test for testing parsing "-source" parameter in Balancer CLIMajortestMingliang LiuMingliang Liu
HDFS-10558DiskBalancer: Print the full path to plan fileMinorbalancer & moverAnu EngineerXiaobing Zhou
YARN-5699Retrospect yarn entity fields which are publishing in events info fields.Major.Rohith Sharma K SRohith Sharma K S
YARN-5145[YARN-3368] Move new YARN UI configuration to HADOOP_CONF_DIRMajor.Wangda TanSunil Govindan
HDFS-11013Correct typos in native erasure coding dump codeTrivialerasure-coding, nativeLászló Bence NagyLászló Bence Nagy
HDFS-10922Adding additional unit tests for Trash (II)MajortestXiaoyu YaoWeiwei Yang
HDFS-9390Block management for maintenance statesMajor.Ming MaMing Ma
HADOOP-13061Refactor erasure codersMajor.Rui LiKai Sasaki
YARN-5741[YARN-3368] Update UI2 documentation for new UI2 pathMajor.Kai SasakiKai Sasaki
HDFS-10906Add unit tests for Trash with HDFS encryption zonesMajorencryptionXiaoyu YaoHanisha Koneru
YARN-5561[Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via RESTMajortimelinereaderRohith Sharma K SRohith Sharma K S
HADOOP-13735ITestS3AFileContextStatistics.testStatistics() failingMinorfs/s3Steve LoughranPieter Reuse
HDFS-10976Report erasure coding policy of EC files in FsckMajornamenodeWei-Chiu ChuangWei-Chiu Chuang
HDFS-10998Add unit tests for HDFS command 'dfsadmin -fetchImage' in HAMajortestXiaobing ZhouXiaobing Zhou
YARN-4911Bad placement policy in FairScheduler causes the RM to crashMajorfairschedulerRay ChiangRay Chiang
YARN-5047Refactor nodeUpdate across schedulersMajorcapacityscheduler, fairscheduler, schedulerRay ChiangRay Chiang
HDFS-8410Add computation time metrics to datanode for ECWorkerMajor.Li BoSammi Chen
HDFS-10975fsck -list-corruptfileblocks does not report corrupt EC filesMajor.Wei-Chiu ChuangTakanobu Asanuma
HADOOP-13727S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider.Minorfs/s3Rajesh BalamohanChris Nauroth
HADOOP-12774s3a should use UGI.getCurrentUser.getShortname() for usernameMajorfs/s3Steve LoughranSteve Loughran
HADOOP-13309Document S3A known limitations in file ownership and permission model.Minorfs/s3Chris NaurothChris Nauroth
HDFS-11011Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'Majorhdfs-clientXiaobing ZhouXiaobing Zhou
HDFS-10638Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode.Majordatanode, fsVirajith JalapartiVirajith Jalaparti
HADOOP-11798Native raw erasure coder in XOR codesMajorioKai ZhengSammi Chen
HADOOP-13614Purge some superfluous/obsolete S3 FS tests that are slowing test runs downMinorfs/s3, testSteve LoughranSteve Loughran
HDFS-11038DiskBalancer: support running multiple commands in single testMajorbalancer & moverXiaobing ZhouXiaobing Zhou
YARN-5772Replace old Hadoop logo with new oneMajoryarn-ui-v2Akira AjisakaAkhil PB
YARN-5500'Master node' link under application tab is brokenCritical.Sumana SathishAkhil PB
YARN-5497Use different color for Undefined and Succeeded for Final State in applications pageTrivial.Yesha VoraAkhil PB
YARN-5490[YARN-3368] Fix various alignment issues and broken breadcrumb link in Node pageMajor.Sunil GovindanAkhil PB
YARN-5779[YARN-3368] Document limits/notes of the new YARN UIMajor.Wangda TanWangda Tan
YARN-5785[YARN-3368] Accessing applications and containers list from Node page is throwing few exceptions in consoleMajoryarn-ui-v2Sunil GovindanAkhil PB
YARN-5799Fix Opportunistic Allocation to set the correct value of Node Http AddressMajorresourcemanagerArun SureshArun Suresh
YARN-4765Split TestHBaseTimelineStorage into multiple test classesMajortimelineserverVarun SaxenaVarun Saxena
YARN-5804New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075Majoryarn-ui-v2Sunil GovindanSunil Govindan
HADOOP-13680fs.s3a.readahead.range to use getLongBytesMajorfs/s3Steve LoughranAbhishek Modi
YARN-5793Trim configuration values in DockerLinuxContainerRuntimeMinornodemanagerTianyin XuTianyin Xu
HDFS-11030TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing)Majordatanode, testMingliang LiuMingliang Liu
HDFS-10997Reduce number of path resolving methodsMajornamenodeDaryn SharpDaryn Sharp
HDFS-10499TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails IntermittentlyMajornamenode, testHanisha KoneruYiqun Lin
HDFS-10633DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.mdMinordocumentationYiqun LinYiqun Lin
HDFS-11065Add space quota tests for heterogenous storagesMajorhdfsXiaobing ZhouXiaobing Zhou
HDFS-11031Add additional unit test for DataNode startup behavior when volumes failMajordatanode, testMingliang LiuMingliang Liu
HDFS-10566Submit plan request should throw exception if Datanode is in non-REGULAR status.MajordatanodeJitendra Nath PandeyXiaobing Zhou
HDFS-11076Add unit test for extended AclsMajortestChen LiangChen Liang
YARN-2995Enhance UI to show cluster resource utilization of various container Execution typesBlockerresourcemanagerSriram RaoKonstantinos Karanasos
YARN-4849[YARN-3368] cleanup code base, integrate web UI related build to mvn, and fix licenses.Major.Wangda TanWangda Tan
HDFS-11085Add unit test for NameNode failing to start when name dir is unwritableMajornamenode, testMingliang LiuXiaobing Zhou
YARN-5802updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policyCriticalcapacity schedulerBibin A ChundattBibin A Chundatt
YARN-5716Add global scheduler interface definition and update CapacityScheduler to use it.MajorcapacityschedulerWangda TanWangda Tan
YARN-5833Add validation to ensure default ports are unique in ConfigurationMajoryarnKonstantinos KaranasosKonstantinos Karanasos
HDFS-11083Add unit test for DFSAdmin -report commandMajorshell, testMingliang LiuXiaobing Zhou
YARN-4329Allow fetching exact reason as to why a submitted app is in ACCEPTED state in Fair SchedulerMajorfairscheduler, resourcemanagerNaganarasimha G RYufei Gu
YARN-4498Application level node labels stats to be available in RESTMajorapi, client, resourcemanagerBibin A ChundattBibin A Chundatt
YARN-5611Provide an API to update lifetime of an application.MajorresourcemanagerRohith Sharma K SRohith Sharma K S
HADOOP-13720Add more info to the msgs printed in AbstractDelegationTokenSecretManager for better supportabilityTrivialcommon, securityYongjun ZhangYongjun Zhang
HDFS-11122TestDFSAdmin#testReportCommand fails due to timed outMinortestYiqun LinYiqun Lin
HDFS-11119Support for parallel checking of StorageLocations on DataNode startupMajordatanodeArpit AgarwalArpit Agarwal
HDFS-10872Add MutableRate metrics for FSNamesystemLock operationsMajornamenodeErik KrogenErik Krogen
HDFS-11105TestRBWBlockInvalidation#testRWRInvalidation fails intermittentlyMajornamenode, testYiqun LinYiqun Lin
HDFS-11114Support for running async disk checks in DataNodeMajordatanodeArpit AgarwalArpit Agarwal
HADOOP-13822Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics()MajorfsBrahma Reddy BattulaBrahma Reddy Battula
HADOOP-11601Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty filesMinorfs, testSteve LoughranSteve Loughran
HDFS-11113Document dfs.client.read.striped configuration in hdfs-default.xmlMinordocumentation, hdfs-clientRakesh RRakesh R
HDFS-11148Update DataNode to use StorageLocationChecker at startupMajordatanodeArpit AgarwalArpit Agarwal
HADOOP-13655document object store use with fs shell and distcpMajordocumentation, fs, fs/s3Steve LoughranSteve Loughran
YARN-5865Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611Major.Sunil GovindanSunil Govindan
HADOOP-13801regression: ITestS3AMiniYarnCluster failingMajorfs/s3Steve LoughranSteve Loughran
HDFS-10368Erasure Coding: Deprecate replication-related config keysMajorerasure-codingRakesh RRakesh R
YARN-5649Add REST endpoints for updating application timeoutsMajorschedulerRohith Sharma K SRohith Sharma K S
YARN-4206Add Application timeouts in Application report and CLIMajorschedulernijelRohith Sharma K S
HDFS-10994Support an XOR policy XOR-2-1-64k in HDFSMajorerasure-codingSammi ChenSammi Chen
HADOOP-13823s3a rename: fail if dest file existsBlockerfs/s3Steve LoughranSteve Loughran
HDFS-11149Support for parallel checking of FsVolumesMajordatanodeArpit AgarwalArpit Agarwal
HDFS-8678Bring back the feature to view chunks of files in the HDFS file browserMajoruiRavi PrakashRavi Prakash
YARN-5761Separate QueueManager from SchedulerMajorcapacityschedulerXuan GongXuan Gong
HADOOP-13857S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEsMajorfs/s3Steve LoughranSteve Loughran
HADOOP-13855Fix a couple of the s3a statistic names to be consistent with the restMajorfs/s3Steve LoughranSteve Loughran
HADOOP-13257Improve Azure Data Lake contract tests.Majorfs/adlChris NaurothVishwajeet Dusane
YARN-5746The state of the parentQueue and its childQueues should be synchronized.Majorcapacity scheduler, resourcemanagerXuan GongXuan Gong
HDFS-11178TestAddStripedBlockInFBR#testAddBlockInFullBlockReport fails frequently in trunkMajortestYiqun LinYiqun Lin
HDFS-11172Support an erasure coding policy using RS 10 + 4Majorerasure-codingSammi ChenWei Zhou
YARN-5965Retrospect ApplicationReport#getApplicationTimeoutsMajorschedulerJian HeRohith Sharma K S
YARN-5922Remove direct references of HBaseTimelineWriter/Reader in core ATS classesMajoryarnHaibo ChenHaibo Chen
HDFS-8630WebHDFS : Support get/set/unset StoragePolicyMajorwebhdfsnijelSurendra Singh Lilhore
YARN-5925Extract hbase-backend-exclusive utility methods from TimelineStorageUtilMajoryarnHaibo ChenHaibo Chen
YARN-5982Simplify opportunistic container parameters and metricsMajor.Konstantinos KaranasosKonstantinos Karanasos
YARN-5970Validate application update timeout request parametersMajorschedulerRohith Sharma K SRohith Sharma K S
HADOOP-13871ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awfulMajorfs/s3Steve LoughranSteve Loughran
YARN-2009CapacityScheduler: Add intra-queue preemption for app priority supportMajorcapacityschedulerDevaraj KSunil Govindan
HDFS-8411Add bytes count metrics to datanode for ECWorkerMajor.Li BoSammi Chen
HADOOP-11804Shaded Hadoop client artifacts and miniclusterMajorbuildSean BusbeySean Busbey
HDFS-11188Change min supported DN and NN versions back to 2.xCriticalrolling upgradesAndrew WangAndrew Wang
YARN-5524Yarn live log aggregation does not throw if command line arg is wrongMajorlog-aggregationPrasanth JayachandranXuan Gong
YARN-5650Render Application Timeout value in web UIMajorschedulerRohith Sharma K SAkhil PB
HDFS-11182Update DataNode to use DatasetVolumeCheckerMajordatanodeArpit AgarwalArpit Agarwal
YARN-5976Update hbase version to 1.2 (removes phoenix dependencies)Major.Vrushali CVrushali C
YARN-4990Re-direction of a particular log file within in a container in NM UI does not redirect properly to Log Server ( history ) on container completionMajor.Hitesh ShahXuan Gong
YARN-5706Fail to launch SLSRunner due to NPEMajorscheduler-load-simulatorKai SasakiKai Sasaki
YARN-5938Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixesMajor.Arun SureshArun Suresh
YARN-5756Add state-machine implementation for scheduler queuesMajor.Xuan GongXuan Gong
YARN-5931Document timeout interfaces CLI and REST APIsMajorschedulerRohith Sharma K SRohith Sharma K S
YARN-5529Create new DiskValidator class with metricsMajornodemanagerRay ChiangYufei Gu
YARN-6025Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementationsMajorcapacity scheduler, schedulerNaganarasimha G RNaganarasimha G R
YARN-5923Unable to access logs for a running application if YARN_ACL_ENABLE is enabledMajor.Xuan GongXuan Gong
YARN-4899Queue metrics of SLS capacity scheduler only activated after app submit to the queueMajorcapacity schedulerWangda TanJonathan Hung
YARN-5906Update AppSchedulingInfo to use SchedulingPlacementSetMajor.Wangda TanWangda Tan
YARN-6009RM fails to start during an upgrade - Failed to load/recover state (YarnException: Invalid application timeout, value=0 for type=LIFETIME)CriticalresourcemanagerGour SahaRohith Sharma K S
YARN-6074FlowRunEntity does not deserialize long values correctlyMajortimelinereaderRohith Sharma K SRohith Sharma K S
YARN-3955Support for application priority ACLs in queues of CapacitySchedulerMajorcapacityschedulerSunil GovindanSunil Govindan
HDFS-11072Add ability to unset and change directory EC policyMajorerasure-codingAndrew WangSammi Chen
HDFS-9391Update webUI/JMX to display maintenance state infoMajor.Ming MaManoj Govindassamy
YARN-5416TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stoppedMinortest, yarnJunping DuJunping Du
HADOOP-13336S3A to support per-bucket configurationMajorfs/s3Steve LoughranSteve Loughran
YARN-6011Add a new web service to list the files on a container in AHSWebServiceMajor.Xuan GongXuan Gong
YARN-6016Fix minor bugs in handling of local AMRMToken in AMRMProxyMinorfederationBotong HuangBotong Huang
YARN-5556CapacityScheduler: Support deleting queues without requiring a RM restartMajorcapacity schedulerXuan GongNaganarasimha G R
HDFS-11259Update fsck to display maintenance state infoMajordatanode, namenodeManoj GovindassamyManoj Govindassamy
YARN-5831Propagate allowPreemptionFrom flag all the way down to the appMajorfairschedulerKarthik KambatlaYufei Gu
YARN-5928Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn serversMajoryarnHaibo ChenHaibo Chen
YARN-5959RM changes to support change of container ExecutionTypeMajor.Arun SureshArun Suresh
YARN-6040Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classesMajor.Wangda TanWangda Tan

OTHER:

JIRASummaryPriorityComponentReporterContributor
HDFS-10861Refactor StripeReaders and use ECChunk version decode APIMajor.Sammi ChenSammi Chen
MAPREDUCE-6780Add support for striping files in benchmarking of TeraGen and TeraSortMajor.Sammi ChenSammi Chen
HDFS-10895Update HDFS Erasure Coding doc to add how to use ISA-L based coderMajor.Sammi ChenSammi Chen
HDFS-10984Expose nntop output as metricsMajornamenodeSiddharth WagleSiddharth Wagle
YARN-5717Add tests for container-executor's is_feature_enabled functionMajoryarnSidharta SeethanaSidharta Seethana
YARN-5704Provide config knobs to control enabling/disabling new/work in progress features in container-executorMajoryarnSidharta SeethanaSidharta Seethana
HDFS-11033Add documents for native raw erasure coder in XOR codesMajordocumentation, erasure-codingSammi ChenSammi Chen
YARN-5308FairScheduler: Move continuous scheduling related tests to TestContinuousSchedulingMajorfairscheduler, testKarthik KambatlaKai Sasaki
YARN-5822Log ContainerRuntime initialization error in LinuxContainerExecutorTrivialnodemanagerSidharta SeethanaSidharta Seethana
HDFS-11145Implement getTrashRoot() for ViewFileSystemMajor.Manoj GovindassamyManoj Govindassamy
HDFS-11058Implement 'hadoop fs -df' command for ViewFileSystemMajor.Manoj GovindassamyManoj Govindassamy
HADOOP-13828Implement getFileChecksum(path, length) for ViewFileSystemMajorviewfsManoj GovindassamyManoj Govindassamy
YARN-4395Typo in comment in ClientServiceDelegateTrivial.Daniel TempletonAlison Yu
MAPREDUCE-6810hadoop-mapreduce-client-nativetask compilation broken on GCC-6.2.1Major.Ravi PrakashRavi Prakash
HADOOP-13840Implement getUsed() for ViewFileSystemMajorviewfsManoj GovindassamyManoj Govindassamy
HDFS-11216Add remoteBytesRead counter metrics for erasure coding reconstruction taskMajor.Sammi ChenSammi Chen
YARN-5719Enforce a C standard for native container-executorMajornodemanagerChris DouglasChris Douglas
HADOOP-13885Implement getLinkTarget for ViewFileSystemMajorviewfsManoj GovindassamyManoj Govindassamy
YARN-5646Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containersBlocker.Konstantinos KaranasosKonstantinos Karanasos
HDFS-9809Abstract implementation-specific details from the datanodeMajordatanode, fsVirajith JalapartiVirajith Jalaparti
HADOOP-13835Move Google Test Framework code from mapreduce to hadoop-commonMajortestVarun VasudevVarun Vasudev