Apache Hadoop Changelog

Release 3.1.3 - 2019-09-12

INCOMPATIBLE CHANGES:

JIRA	Summary	Priority	Component	Reporter	Contributor
HADOOP-15922	DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL	Major	common, kms	He Xiaoqiao	He Xiaoqiao

NEW FEATURES:

JIRA	Summary	Priority	Component	Reporter	Contributor
HADOOP-15950	Failover for LdapGroupsMapping	Major	common, security	Lukas Majercak	Lukas Majercak

IMPROVEMENTS:

JIRA	Summary	Priority	Component	Reporter	Contributor
HADOOP-15481	Emit FairCallQueue stats as metrics	Major	metrics, rpc-server	Erik Krogen	Christopher Gregorian
HDFS-14213	Remove Jansson from BUILDING.txt	Minor	documentation	Akira Ajisaka	Dinesh Chitlangia
HDFS-14221	Replace Guava Optional with Java Optional	Major	.	Arpit Agarwal	Arpit Agarwal
HDFS-14222	Make ThrottledAsyncChecker constructor public	Major	.	Arpit Agarwal	Arpit Agarwal
HADOOP-16089	AliyunOSS: update oss-sdk version to 3.4.1	Major	fs/oss	wujinhu	wujinhu
HDFS-14231	DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error	Major	hdfs	Kitti Nanasi	Kitti Nanasi
YARN-7171	RM UI should sort memory / cores numerically	Major	.	Eric Maynard	Ahmed Hussein
YARN-9282	Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'	Trivial	.	Szilard Nemeth	Charan Hebri
HADOOP-16108	Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's	Major	.	Harshakiran Reddy	Ayush Saxena
YARN-8295	[UI2] Improve "Resource Usage" tab error message when there are no data available.	Minor	yarn-ui-v2	Gergely Novák	Charan Hebri
YARN-7824	[UI2] Yarn Component Instance page should include link to container logs	Major	yarn-ui-v2	Yesha Vora	Akhil PB
HADOOP-15281	Distcp to add no-rename copy option	Major	tools/distcp	Steve Loughran	Andrew Olson
YARN-9309	Improve graph text in SLS to avoid overlapping	Minor	.	Bilwa S T	Bilwa S T
YARN-9168	DistributedShell client timeout should be -1 by default	Minor	.	Zhankun Tang	Zhankun Tang
YARN-9087	Improve logging for initialization of Resource plugins	Major	yarn	Szilard Nemeth	Szilard Nemeth
YARN-9121	Replace GpuDiscoverer.getInstance() to a readable object for easy access control	Major	.	Szilard Nemeth	Szilard Nemeth
YARN-9139	Simplify initializer code of GpuDiscoverer	Major	.	Szilard Nemeth	Szilard Nemeth
HDFS-14247	Repeat adding node description into network topology	Minor	datanode	HuangTao	HuangTao
YARN-9138	Improve test coverage for nvidia-smi binary execution of GpuDiscoverer	Major	.	Szilard Nemeth	Szilard Nemeth
MAPREDUCE-7191	JobHistoryServer should log exception when loading/parsing history file failed	Minor	mrv2	Jiandan Yang	Jiandan Yang
HDFS-14346	Better time precision in getTimeDuration	Minor	namenode	Chao Sun	Chao Sun
HDFS-14366	Improve HDFS append performance	Major	hdfs	Chao Sun	Chao Sun
MAPREDUCE-7190	Add SleepJob additional parameter to make parallel runs distinguishable	Major	.	Adam Antal	Adam Antal
HADOOP-16208	Do Not Log InterruptedException in Client	Minor	common	David Mollitor	David Mollitor
YARN-9463	Add queueName info when failing with queue capacity sanity check	Trivial	capacity scheduler	Aihua Xu	Aihua Xu
HADOOP-16227	Upgrade checkstyle to 8.19	Major	build	Akira Ajisaka	Akira Ajisaka
HDFS-14432	dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml	Minor	hdfs	puleya7	puleya7
HDFS-14463	Add Log Level link under NameNode and DataNode Web UI Utilities dropdown	Trivial	webhdfs	Siyao Meng	Siyao Meng
YARN-9529	Log correct cpu controller path on error while initializing CGroups.	Major	nodemanager	Jonathan Hung	Jonathan Hung
HADOOP-16289	Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh	Major	scripts	Siyao Meng	Siyao Meng
HADOOP-16307	Intern User Name and Group Name in FileStatus	Major	fs	David Mollitor	David Mollitor
HDFS-14507	Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles	Minor	documentation	Siyao Meng	Siyao Meng
HDFS-14451	Incorrect header or version mismatch log message	Minor	ipc	David Mollitor	Shweta
HDFS-14502	keepResults option in NNThroughputBenchmark should call saveNamespace()	Major	benchmarks, hdfs	Konstantin Shvachko	Konstantin Shvachko
HADOOP-16323	https everywhere in Maven settings	Minor	build	Akira Ajisaka	Akira Ajisaka
YARN-9563	Resource report REST API could return NaN or Inf	Minor	.	Ahmed Hussein	Ahmed Hussein
YARN-9545	Create healthcheck REST endpoint for ATSv2	Major	ATSv2	Zoltan Siegl	Zoltan Siegl
HDFS-10659	Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory	Major	ha, journal-node	Amit Anand	star
HDFS-14513	FSImage which is saving should be clean while NameNode shutdown	Major	namenode	He Xiaoqiao	He Xiaoqiao
YARN-9543	[UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2	Major	ATSv2, yarn-ui-v2	Zoltan Siegl	Zoltan Siegl
HADOOP-16369	Fix zstandard shortname misspelled as zts	Major	.	Jonathan Eagles	Jonathan Eagles
HDFS-14560	Allow block replication parameters to be refreshable	Major	namenode	Stephen O'Donnell	Stephen O'Donnell
HDFS-12770	Add doc about how to disable client socket cache	Trivial	hdfs-client	Weiwei Yang	Weiwei Yang
HADOOP-9157	Better option for curl in hadoop-auth-examples	Minor	documentation	Jingguo Yao	Andras Bokor
HDFS-14340	Lower the log level when can't get postOpAttr	Minor	nfs	Anuhan Torgonshar	Anuhan Torgonshar
HADOOP-15914	hadoop jar command has no help argument	Major	common	Adam Antal	Adam Antal
HADOOP-16156	[Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl	Minor	.	Shweta	Shweta
HADOOP-14385	HttpExceptionUtils#validateResponse swallows exceptions	Trivial	.	Wei-Chiu Chuang	Wei-Chiu Chuang
HDFS-12564	Add the documents of swebhdfs configurations on the client side	Major	documentation, webhdfs	Takanobu Asanuma	Takanobu Asanuma
HDFS-14403	Cost-Based RPC FairCallQueue	Major	ipc, namenode	Erik Krogen	Christopher Gregorian
HADOOP-16266	Add more fine-grained processing time metrics to the RPC layer	Minor	ipc	Christopher Gregorian	Erik Krogen
YARN-9629	Support configurable MIN_LOG_ROLLING_INTERVAL	Minor	log-aggregation, nodemanager, yarn	Adam Antal	Adam Antal
HDFS-13694	Making md5 computing being in parallel with image loading	Major	.	zhouyingchao	Lisheng Sun
HDFS-14632	Reduce useless #getNumLiveDataNodes call in SafeModeMonitor	Major	namenode	He Xiaoqiao	He Xiaoqiao
YARN-9573	DistributedShell cannot specify LogAggregationContext	Major	distributed-shell, log-aggregation, yarn	Adam Antal	Adam Antal
YARN-9337	GPU auto-discovery script runs even when the resource is given by hand	Major	yarn	Adam Antal	Adam Antal
YARN-9127	Create more tests to verify GpuDeviceInformationParser	Major	.	Szilard Nemeth	Peter Bacsko
HDFS-14547	DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.	Major	.	Jinglun	Jinglun
HDFS-14697	Backport HDFS-14513 to branch-2	Minor	namenode	He Xiaoqiao	He Xiaoqiao
YARN-8045	Reduce log output from container status calls	Major	.	Shane Kumpf	Craig Condit
HDFS-14693	NameNode should log a warning when EditLog IPC logger's pending size exceeds limit.	Minor	namenode	Xudong Cao	Xudong Cao
YARN-9094	Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM	Trivial	.	Szilard Nemeth	Gergely Pollak
YARN-9096	Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily	Major	.	Szilard Nemeth	Gergely Pollak
YARN-9092	Create an object for cgroups mount enable and cgroups mount path as they belong together	Minor	.	Szilard Nemeth	Gergely Pollak
YARN-9124	Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently	Minor	.	Szilard Nemeth	Adam Antal
YARN-8199	Logging fileSize of log files under NM Local Dir	Major	log-aggregation	Prabhu Joseph	Prabhu Joseph
YARN-9729	[UI2] Fix error message for logs when ATSv2 is offline	Major	yarn-ui-v2	Zoltan Siegl	Zoltan Siegl
YARN-9135	NM State store ResourceMappings serialization are tested with Strings instead of real Device objects	Major	.	Szilard Nemeth	Peter Bacsko
HDFS-14370	Edit log tailing fast-path should allow for backoff	Major	namenode, qjm	Erik Krogen	Erik Krogen
YARN-9442	container working directory has group read permissions	Minor	yarn	Jim Brennan	Jim Brennan
HADOOP-16459	Backport [HADOOP-16266] "Add more fine-grained processing time metrics to the RPC layer" to branch-2	Major	.	Erik Krogen	Erik Krogen
HDFS-14491	More Clarity on Namenode UI Around Blocks and Replicas	Minor	.	Alan Jackoway	Siyao Meng
YARN-9140	Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager	Trivial	.	Szilard Nemeth	Peter Bacsko
YARN-9488	Skip YARNFeatureNotEnabledException from ClientRMService	Minor	resourcemanager	Prabhu Joseph	Prabhu Joseph
YARN-8586	Extract log aggregation related fields and methods from RMAppImpl	Major	.	Szilard Nemeth	Peter Bacsko
YARN-9100	Add tests for GpuResourceAllocator and do minor code cleanup	Major	.	Szilard Nemeth	Peter Bacsko
HADOOP-15246	SpanReceiverInfo - Prefer ArrayList over LinkedList	Trivial	common	David Mollitor	David Mollitor
HADOOP-16158	DistCp to support checksum validation when copy blocks in parallel	Major	tools/distcp	Kai Xie	Kai Xie
HDFS-14746	Trivial test code update after HDFS-14687	Trivial	ec	Wei-Chiu Chuang	kevin su
HDFS-13709	Report bad block to NN when transfer block encounter EIO exception	Major	datanode	Chen Zhang	Chen Zhang
HDFS-14665	HttpFS: LISTSTATUS response is missing HDFS-specific fields	Major	httpfs	Siyao Meng	Siyao Meng
HDFS-14276	[SBN read] Reduce tailing overhead	Major	ha, namenode	Wei-Chiu Chuang	Ayush Saxena
HDFS-14748	Make DataNodePeerMetrics#minOutlierDetectionSamples configurable	Major	.	Lisheng Sun	Lisheng Sun
HADOOP-15998	Ensure jar validation works on Windows.	Blocker	build	Brian Grunkemeyer	Brian Grunkemeyer
HDFS-14633	The StorageType quota and consume in QuotaFeature is not handled for rename	Major	.	Jinglun	Jinglun
YARN-9795	ClusterMetrics to include AM allocation delay	Minor	.	Fengnan Li	Fengnan Li
YARN-8995	Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time.	Major	metrics, nodemanager, resourcemanager	zhuqi	zhuqi

BUG FIXES:

JIRA	Summary	Priority	Component	Reporter	Contributor
HDFS-13642	Creating a file with block size smaller than EC policy's cell size should fail	Major	erasure-coding	Xiao Chen	Xiao Chen
HADOOP-15948	Inconsistency in get and put syntax if filename/dirname contains space	Minor	fs	vivek kumar	Ayush Saxena
HDFS-13816	dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException	Major	namenode	Vinayakumar B	Vinayakumar B
HADOOP-15966	Hadoop Kerberos broken on macos as java.security.krb5.realm is reset: Null realm name (601)	Major	scripts	Steve Loughran	Steve Loughran
HADOOP-16028	Fix NetworkTopology chooseRandom function to support excluded nodes	Major	.	Sihai Ke	Sihai Ke
YARN-9162	Fix TestRMAdminCLI#testHelp	Major	resourcemanager, test	Ayush Saxena	Ayush Saxena
HADOOP-16031	TestSecureLogins#testValidKerberosName fails	Major	security	Akira Ajisaka	Akira Ajisaka
HADOOP-16016	TestSSLFactory#testServerWeakCiphers sporadically fails in precommit builds	Major	security, test	Jason Lowe	Akira Ajisaka
HDFS-14198	Upload and Create button doesn't get enabled after getting reset.	Major	.	Ayush Saxena	Ayush Saxena
YARN-9203	Fix typos in yarn-default.xml	Trivial	documentation	Rahul Padmanabhan	Rahul Padmanabhan
HDFS-14207	ZKFC should catch exception when ha configuration missing	Major	hdfs	Fei Hui	Fei Hui
HDFS-14218	EC: Ls -e throw NPE when directory ec policy is disabled	Major	.	Surendra Singh Lilhore	Ayush Saxena
YARN-9210	RM nodes web page can not display node info	Blocker	yarn	Jiandan Yang	Jiandan Yang
YARN-8961	[UI2] Flow Run End Time shows 'Invalid date'	Major	.	Charan Hebri	Akhil PB
YARN-7088	Add application launch time to Resource Manager REST API	Major	.	Abdullah Yousufi	Kanwaljeet Sachdev
YARN-9222	Print launchTime in ApplicationSummary	Major	.	Jonathan Hung	Jonathan Hung
YARN-8901	Restart "NEVER" policy does not work with component dependency	Critical	.	Yesha Vora	Suma Shivaprasad
YARN-9237	NM should ignore sending finished apps to RM during RM fail-over	Major	yarn	Jiandan Yang	Jiandan Yang
YARN-6616	YARN AHS shows submitTime for jobs same as startTime	Minor	.	Prabhu Joseph	Prabhu Joseph
YARN-9099	GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way	Major	.	Szilard Nemeth	Szilard Nemeth
HADOOP-16086	Backport HADOOP-15549 to branch-3.1	Major	metrics	Yuming Wang	Todd Lipcon
YARN-9206	RMServerUtils does not count SHUTDOWN as an accepted state	Major	.	Kuhu Shukla	Kuhu Shukla
HADOOP-16096	HADOOP-15281/distcp -Xdirect needs to use commons-logging on 3.1	Critical	.	Eric Payne	Steve Loughran
HDFS-14140	JournalNodeSyncer authentication is failing in secure cluster	Major	journal-node, security	Surendra Singh Lilhore	Surendra Singh Lilhore
YARN-9257	Distributed Shell client throws a NPE for a non-existent queue	Major	distributed-shell	Charan Hebri	Charan Hebri
YARN-8761	Service AM support for decommissioning component instances	Major	.	Billie Rinaldi	Billie Rinaldi
HDFS-14266	EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available.	Major	.	Harshakiran Reddy	Ayush Saxena
HDFS-14274	EC: NPE While Listing EC Policy For A Directory Following Replication Policy.	Major	erasure-coding	Souryakanta Dwivedy	Ayush Saxena
HDFS-14263	Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream()	Major	datanode	Surendra Singh Lilhore	Surendra Singh Lilhore
YARN-7761	[UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link	Major	yarn-ui-v2	Sumana Sathish	Akhil PB
YARN-9295	[UI2] Fix label typo in Cluster Overview page	Trivial	yarn-ui-v2	Charan Hebri	Charan Hebri
YARN-9284	Fix the unit of yarn.service.am-resource.memory in the document	Minor	documentation, yarn-native-services	Masahiro Tanaka	Masahiro Tanaka
YARN-9283	Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference	Minor	documentation	Szilard Nemeth	Adam Antal
YARN-9286	[Timeline Server] Sorting based on FinalStatus shows pop-up message	Minor	timelineserver	Nallasivan	Bilwa S T
HDFS-14081	hdfs dfsadmin -metasave metasave_test results NPE	Major	hdfs	Shweta	Shweta
HADOOP-15813	Enable more reliable SSL connection reuse	Major	common	Daryn Sharp	Daryn Sharp
HADOOP-16105	WASB in secure mode does not set connectingUsingSAS	Major	fs/azure	Steve Loughran	Steve Loughran
YARN-9238	Avoid allocating opportunistic containers to previous/removed/non-exist application attempt	Critical	.	lujie	lujie
YARN-9118	Handle exceptions with parsing user defined GPU devices in GpuDiscoverer	Major	.	Szilard Nemeth	Szilard Nemeth
YARN-9317	Avoid repeated YarnConfiguration#timelineServiceV2Enabled check	Major	.	Bibin A Chundatt	Prabhu Joseph
YARN-9213	RM Web UI v1 does not show custom resource allocations for containers page	Major	.	Szilard Nemeth	Szilard Nemeth
YARN-9248	RMContainerImpl:Invalid event: ACQUIRED at KILLED	Major	.	lujie	lujie
HADOOP-16018	DistCp won't reassemble chunks when blocks per chunk > 0	Major	tools/distcp	Kai Xie	Kai Xie
YARN-9334	YARN Service Client does not work with SPNEGO when knox is configured	Major	yarn-native-services	Tarun Parimi	Billie Rinaldi
HDFS-14305	Serial number in BlockTokenSecretManager could overlap between different namenodes	Major	namenode, security	Chao Sun	He Xiaoqiao
HDFS-14314	fullBlockReportLeaseId should be reset after registering to NN	Critical	datanode	star	star
YARN-8803	[UI2] Show flow runs in the order of recently created time in graph widgets	Major	yarn-ui-v2	Akhil PB	Akhil PB
HADOOP-16114	NetUtils#canonicalizeHost gives different value for same host	Minor	net	Praveen Krishna	Praveen Krishna
HDFS-14317	Standby does not trigger edit log rolling when in-progress edit log tailing is enabled	Critical	.	Ekanth Sethuramalingam	Ekanth Sethuramalingam
HDFS-14333	Datanode fails to start if any disk has errors during Namenode registration	Major	datanode	Stephen O'Donnell	Stephen O'Donnell
HADOOP-16192	CallQueue backoff bug fixes: doesn't perform backoff when add() is used, and doesn't update backoff when refreshed	Major	ipc	Erik Krogen	Erik Krogen
HDFS-14037	Fix SSLFactory truststore reloader thread leak in URLConnectionFactory	Major	hdfs-client, webhdfs	Takanobu Asanuma	Takanobu Asanuma
HADOOP-16225	Fix links to the developer mailing lists in DownstreamDev.md	Minor	documentation	Akira Ajisaka	Wanqiang Ji
HADOOP-16232	Fix errors in the checkstyle configration xmls	Major	build	Akira Ajisaka	Wanqiang Ji
HDFS-14389	getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured	Major	namenode	Stephen O'Donnell	Stephen O'Donnell
HDFS-14407	Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes	Minor	.	Wanqiang Ji	Wanqiang Ji
YARN-9413	Queue resource leak after app fail for CapacityScheduler	Major	capacityscheduler	Tao Yang	Tao Yang
HADOOP-14544	DistCp documentation for command line options is misaligned.	Minor	documentation	Chris Nauroth	Masatake Iwasaki
HDFS-10477	Stop decommission a rack of DataNodes caused NameNode fail over to standby	Major	namenode	yunjiong zhao	yunjiong zhao
YARN-6695	Race condition in RM for publishing container events vs appFinished events causes NPE	Critical	.	Rohith Sharma K S	Prabhu Joseph
YARN-8622	NodeManager native build fails due to getgrouplist not found on macOS	Major	nodemanager	Ewan Higgs	Siyao Meng
HADOOP-16265	Configuration#getTimeDuration is not consistent between default value and manual settings.	Major	.	star	star
YARN-9307	node_partitions constraint does not work	Major	.	kyungwan nam	kyungwan nam
HDFS-13677	Dynamic refresh Disk configuration results in overwriting VolumeMap	Blocker	.	xuzq	xuzq
YARN-9285	RM UI progress column is of wrong type	Minor	yarn	Ahmed Hussein	Ahmed Hussein
HADOOP-16278	With S3A Filesystem, Long Running services End up Doing lot of GC and eventually die	Major	common, hadoop-aws, metrics	Rajat Khandelwal	Rajat Khandelwal
YARN-9504	[UI2] Fair scheduler queue view page does not show actual capacity	Major	fairscheduler, yarn-ui-v2	Zoltan Siegl	Zoltan Siegl
YARN-9519	TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config	Major	log-aggregation	Adam Antal	Adam Antal
HADOOP-16247	NPE in FsUrlConnection	Major	hdfs-client	Karthik Palanisamy	Karthik Palanisamy
HADOOP-16248	MutableQuantiles leak memory under heavy load	Major	metrics	Alexis Daboville	Alexis Daboville
HDFS-14323	Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path	Major	webhdfs	Srinivasu Majeti	Srinivasu Majeti
MAPREDUCE-7205	Treat container scheduler kill exit code as a task attempt killing event	Major	applicationmaster, mr-am, mrv2	Wanqiang Ji	Wanqiang Ji
HDFS-14500	NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished	Major	namenode	Erik Krogen	Erik Krogen
HADOOP-16331	Fix ASF License check in pom.xml	Major	.	Wanqiang Ji	Akira Ajisaka
YARN-9542	Fix LogsCLI guessAppOwner ignores custom file format suffix	Minor	log-aggregation	Prabhu Joseph	Prabhu Joseph
HDFS-14512	ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes)	Major	.	Shen Yinjie	Ayush Saxena
HADOOP-16334	Fix yetus-wrapper not working when HADOOP_YETUS_VERSION >= 0.9.0	Major	yetus	Wanqiang Ji	Wanqiang Ji
HDFS-14521	Suppress setReplication logging.	Major	.	Kihwal Lee	Kihwal Lee
YARN-9507	Fix NPE in NodeManager#serviceStop on startup failure	Minor	.	Bilwa S T	Bilwa S T
YARN-8947	[UI2] Active User info missing from UI2	Major	yarn-ui-v2	Akhil PB	Akhil PB
YARN-8906	[UI2] NM hostnames not displayed correctly in Node Heatmap Chart	Major	.	Charan Hebri	Akhil PB
YARN-8625	Aggregate Resource Allocation for each job is not present in ATS	Major	ATSv2	Prabhu Joseph	Prabhu Joseph
HADOOP-16345	Potential NPE when instantiating FairCallQueue metrics	Major	ipc	Erik Krogen	Erik Krogen
YARN-9594	Fix missing break statement in ContainerScheduler#handle	Major	.	lujie	lujie
YARN-9565	RMAppImpl#ranNodes not cleared on FinalTransition	Major	.	Bibin A Chundatt	Bilwa S T
YARN-9547	ContainerStatusPBImpl default execution type is not returned	Major	.	Bibin A Chundatt	Bilwa S T
HDFS-13231	Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI	Major	datanode, namenode	Haibo Yan	Stephen O'Donnell
YARN-9621	FIX TestDSWithMultipleNodeManager.testDistributedShellWithPlacementConstraint on branch-3.1	Major	distributed-shell, test	Peter Bacsko	Prabhu Joseph
HDFS-14535	The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read	Major	hdfs-client	Zheng Hu	Zheng Hu
HDFS-13730	BlockReaderRemote.sendReadResult throws NPE	Major	hdfs-client	Wei-Chiu Chuang	Yuanbo Liu
YARN-9584	Should put initializeProcessTrees method call before get pid	Critical	nodemanager	Wanqiang Ji	Wanqiang Ji
HDFS-14010	Pass correct DF usage to ReservedSpaceCalculator builder	Minor	.	Lukas Majercak	Lukas Majercak
HDFS-14078	Admin helper fails to prettify NullPointerExceptions	Major	.	Elek, Marton	Elek, Marton
HDFS-14101	Random failure of testListCorruptFilesCorruptedBlock	Major	test	Kihwal Lee	Zsolt Venczel
HDFS-14465	When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit.	Major	.	Yicong Cai	Yicong Cai
HDFS-12487	FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers	Major	balancer & mover, diskbalancer	liumi	liumi
HDFS-14074	DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace.	Major	hdfs	guangyi lu	guangyi lu
HDFS-14541	When evictableMmapped or evictable size is zero, do not throw NoSuchElementException	Major	hdfs-client, performance	Zheng Hu	Lisheng Sun
HDFS-14598	Findbugs warning caused by HDFS-12487	Minor	diskbalancer	Wei-Chiu Chuang	He Xiaoqiao
YARN-9639	DecommissioningNodesWatcher cause memory leak	Blocker	.	Bibin A Chundatt	Bilwa S T
YARN-9327	Improve synchronisation in ProtoUtils#convertToProtoFormat block	Critical	.	Bibin A Chundatt	Bibin A Chundatt
YARN-9655	AllocateResponse in FederationInterceptor lost applicationPriority	Major	federation	hunshenshi	hunshenshi
HADOOP-16385	Namenode crashes with "RedundancyMonitor thread received Runtime exception"	Major	.	krishna reddy	Ayush Saxena
YARN-9644	First RMContext object is always leaked during switch over	Blocker	.	Bibin A Chundatt	Bibin A Chundatt
HDFS-14629	Property value Hard Coded in DNConf.java	Trivial	.	hemanthboyina	hemanthboyina
YARN-9557	Application fails in diskchecker when ReadWriteDiskValidator is configured.	Critical	nodemanager	Anuruddh Nayak	Bilwa S T
HDFS-12703	Exceptions are fatal to decommissioning monitor	Critical	namenode	Daryn Sharp	He Xiaoqiao
HDFS-12748	NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY	Major	hdfs	Jiandan Yang	Weiwei Yang
YARN-9625	UI2 - No link to a queue on the Queues page for Fair Scheduler	Major	.	Charan Hebri	Zoltan Siegl
HDFS-14466	Add a regression test for HDFS-14323	Minor	fs, test, webhdfs	Yuya Ebihara	Masatake Iwasaki
YARN-9235	If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown	Major	yarn	Antal Bálint Steinbach	Adam Antal
YARN-9626	UI2 - Fair scheduler queue apps page issues	Major	.	Charan Hebri	Zoltan Siegl
YARN-9682	Wrong log message when finalizing the upgrade	Trivial	.	kyungwan nam	kyungwan nam
HADOOP-16440	Distcp can not preserve timestamp with -delete option	Major	.	ludun	ludun
MAPREDUCE-7076	TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build	Minor	test	Rushabh S Shah	kevin su
YARN-9668	UGI conf doesn't read user overridden configurations on RM and NM startup	Major	.	Jonathan Hung	Jonathan Hung
HADOOP-9844	NPE when trying to create an error message response of SASL RPC	Major	ipc	Steve Loughran	Steve Loughran
HADOOP-16245	Enabling SSL within LdapGroupsMapping can break system SSL configs	Major	common, security	Erik Krogen	Erik Krogen
HDFS-14429	Block remain in COMMITTED but not COMPLETE caused by Decommission	Major	.	Yicong Cai	Yicong Cai
HADOOP-16435	RpcMetrics should not be retained forever	Critical	rpc-server	Zoltan Haindrich	Zoltan Haindrich
YARN-9596	QueueMetrics has incorrect metrics when labelled partitions are involved	Major	capacity scheduler	Muhammad Samir Khan	Muhammad Samir Khan
MAPREDUCE-7225	Fix broken current folder expansion during MR job start	Major	mrv2	Adam Antal	Peter Bacsko
HDFS-13529	Fix default trash policy emptier trigger time correctly	Major	namenode	He Xiaoqiao	He Xiaoqiao
HADOOP-15681	AuthenticationFilter should generate valid date format for Set-Cookie header regardless of default Locale	Minor	security	Cao Manh Dat	Cao Manh Dat
HDFS-14685	DefaultAuditLogger doesn't print CallerContext	Major	hdfs	xuzq	xuzq
HDFS-14462	WebHDFS throws "Error writing request body to server" instead of DSQuotaExceededException	Major	webhdfs	Erik Krogen	Simbarashe Dzinamarira
HDFS-14557	JournalNode error: Can't scan a pre-transactional edit log	Major	ha	Wei-Chiu Chuang	Stephen O'Donnell
HDFS-14692	Upload button should not encode complete url	Major	.	Lokesh Jain	Lokesh Jain
HDFS-14631	The DirectoryScanner doesn't fix the wrongly placed replica.	Major	.	Jinglun	Jinglun
YARN-9685	NPE when rendering the info table of leaf queue in non-accessible partitions	Major	capacityscheduler	Tao Yang	Tao Yang
HDFS-14459	ClosedChannelException silently ignored in FsVolumeList.addBlockPool()	Major	datanode	Stephen O'Donnell	Stephen O'Donnell
HDFS-13359	DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream	Major	datanode	Yiqun Lin	Yiqun Lin
YARN-9451	AggregatedLogsBlock shows wrong NM http port	Minor	nodemanager	Prabhu Joseph	Prabhu Joseph
YARN-9723	ApplicationPlacementContext is not required for terminated jobs during recovery	Major	resourcemanager	Prabhu Joseph	Prabhu Joseph
HDFS-12914	Block report leases cause missing blocks until next report	Critical	namenode	Daryn Sharp	Santosh Marella
HDFS-14148	HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory	Major	hdfs	Siyao Meng	Siyao Meng
HDFS-14595	HDFS-11848 breaks API compatibility	Blocker	.	Wei-Chiu Chuang	Siyao Meng
HDFS-14423	Percent (%) and plus (+) characters no longer work in WebHDFS	Major	webhdfs	Jing Wang	Masatake Iwasaki
MAPREDUCE-7230	TestHSWebApp.testLogsViewSingle fails	Major	jobhistoryserver, test	Prabhu Joseph	Prabhu Joseph
HDFS-14687	Standby Namenode never come out of safemode when EC files are being written.	Critical	ec, namenode	Surendra Singh Lilhore	Surendra Singh Lilhore
HDFS-13101	Yet another fsimage corruption related to snapshot	Major	snapshots	Yongjun Zhang	Shashikant Banerjee
HDFS-13201	Fix prompt message in testPolicyAndStateCantBeNull	Minor	.	chencan	chencan
HDFS-14311	Multi-threading conflict at layoutVersion when loading block pool storage	Major	rolling upgrades	Yicong Cai	Yicong Cai
HDFS-14582	Failed to start DN with ArithmeticException when NULL checksum used	Major	datanode	Surendra Singh Lilhore	Surendra Singh Lilhore
HADOOP-16494	Add SHA-256 or SHA-512 checksum to release artifacts to comply with the release distribution policy	Blocker	build	Akira Ajisaka	Akira Ajisaka
YARN-9774	Fix order of arguments for assertEquals in TestSLSUtils	Minor	test	Nikhil Navadiya	Nikhil Navadiya
HDFS-13596	NN restart fails after RollingUpgrade from 2.x to 3.x	Blocker	hdfs	Hanisha Koneru	Fei Hui
HDFS-14396	Failed to load image from FSImageFile when downgrade from 3.x to 2.x	Blocker	rolling upgrades	Fei Hui	Fei Hui
YARN-9642	Fix Memory Leak in AbstractYarnScheduler caused by timer	Blocker	resourcemanager	Bibin A Chundatt	Bibin A Chundatt
HDFS-13977	NameNode can kill itself if it tries to send too many txns to a QJM simultaneously	Major	namenode, qjm	Erik Krogen	Erik Krogen
YARN-9438	launchTime not written to state store for running applications	Major	.	Jonathan Hung	Jonathan Hung
HDFS-12212	Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified	Major	namenode	Vinayakumar B	Vinayakumar B
HDFS-8178	QJM doesn't move aside stale inprogress edits files	Major	qjm	Zhe Zhang	Istvan Fajth
HDFS-14706	Checksums are not checked if block meta file is less than 7 bytes	Major	.	Stephen O'Donnell	Stephen O'Donnell
YARN-9797	LeafQueue#activateApplications should use resourceCalculator#fitsIn	Blocker	.	Bibin A Chundatt	Bilwa S T
YARN-9785	Fix DominantResourceCalculator when one resource is zero	Blocker	.	Bilwa S T	Bilwa S T
YARN-9817	Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero	Major	test	Prabhu Joseph	Prabhu Joseph

TESTS:

JIRA	Summary	Priority	Component	Reporter	Contributor
YARN-9315	TestCapacitySchedulerMetrics fails intermittently	Minor	capacity scheduler	Prabhu Joseph	Prabhu Joseph
YARN-9316	TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently	Minor	capacity scheduler	Prabhu Joseph	Prabhu Joseph
YARN-9325	TestQueueManagementDynamicEditPolicy fails intermittent	Minor	capacity scheduler	Prabhu Joseph	Prabhu Joseph
HDFS-11950	Disable libhdfs zerocopy test on Mac	Minor	libhdfs	John Zhuge	Akira Ajisaka

SUB-TASKS:

JIRA	Summary	Priority	Component	Reporter	Contributor
HADOOP-16045	Don't run TestDU on Windows	Trivial	common, test	Lukas Majercak	Lukas Majercak
HADOOP-16079	Token.toString faulting if any token listed can't load.	Blocker	security	Steve Loughran	Steve Loughran
YARN-9253	Add UT to verify Placement Constraint in Distributed Shell	Major	.	Prabhu Joseph	Prabhu Joseph
YARN-9293	Optimize MockAMLauncher event handling	Major	.	Bibin A Chundatt	Bibin A Chundatt
HADOOP-16109	Parquet reading S3AFileSystem causes EOF	Blocker	fs/s3	Dave Christianson	Steve Loughran
HADOOP-16191	AliyunOSS: improvements for copyFile/copyDirectory and logging	Major	fs/oss	wujinhu	wujinhu
YARN-9391	Disable PATH variable to be passed to Docker container	Major	.	Eric Yang	Jim Brennan
HADOOP-16220	Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project	Major	.	Gabor Bota	Gabor Bota
HADOOP-16233	S3AFileStatus to declare that isEncrypted() is always true	Minor	fs/s3	Steve Loughran	Steve Loughran
HADOOP-16306	AliyunOSS: Remove temporary files when upload small files to OSS	Major	fs/oss	wujinhu	wujinhu
HDFS-14553	Make queue size of BlockReportProcessingThread configurable	Major	namenode	He Xiaoqiao	He Xiaoqiao
HDFS-14034	Support getQuotaUsage API in WebHDFS	Major	fs, webhdfs	Erik Krogen	Chao Sun
YARN-9765	SLS runner crashes when run with metrics turned off.	Major	.	Abhishek Modi	Abhishek Modi
HDFS-14674	[SBN read] Got an unexpected txid when tail editlog	Blocker	.	wangzhaohui	wangzhaohui
YARN-9775	RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore	Major	restapi	Prabhu Joseph	Prabhu Joseph
HDFS-14779	Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns	Major	.	Jonathan Hung	Jonathan Hung

OTHER:

JIRA	Summary	Priority	Component	Reporter	Contributor
HADOOP-16025	Update the year to 2019	Major	build	Ayush Saxena	Ayush Saxena
HDFS-12729	Document special paths in HDFS	Major	documentation	Chris Douglas	Masatake Iwasaki
YARN-9191	Add cli option in DS to support enforceExecutionType in resource requests.	Major	.	Abhishek Modi	Abhishek Modi
HADOOP-16263	Update BUILDING.txt with macOS native build instructions	Minor	.	Siyao Meng	Siyao Meng
YARN-9559	Create AbstractContainersLauncher for pluggable ContainersLauncher logic	Major	.	Jonathan Hung	Jonathan Hung
HADOOP-16551	The changelog*.md seems not generated when create-release	Blocker	.	Zhankun Tang