Apache Hadoop Changelog

Release 3.0.3 - 2018-05-31

INCOMPATIBLE CHANGES:

JIRA	Summary	Priority	Component	Reporter	Contributor
YARN-7190	Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath	Major	timelineclient, timelinereader, timelineserver	Vrushali C	Varun Saxena
HDFS-13099	RBF: Use the ZooKeeper as the default State Store	Minor	documentation	Yiqun Lin	Yiqun Lin

NEW FEATURES:

JIRA	Summary	Priority	Component	Reporter	Contributor
HDFS-13283	Percentage based Reserved Space Calculation for DataNode	Major	datanode, hdfs	Lukas Majercak	Lukas Majercak

IMPROVEMENTS:

JIRA	Summary	Priority	Component	Reporter	Contributor
HDFS-12455	WebHDFS - Adding "snapshot enabled" status to ListStatus query result.	Major	snapshots, webhdfs	Ajay Kumar	Ajay Kumar
HDFS-13062	Provide support for JN to use separate journal disk per namespace	Major	federation, journal-node	Bharat Viswanadham	Bharat Viswanadham
HDFS-12933	Improve logging when DFSStripedOutputStream failed to write some blocks	Minor	erasure-coding	Xiao Chen	chencan
HADOOP-13972	ADLS to support per-store configuration	Major	fs/adl	John Zhuge	Sharad Sonker
YARN-7813	Capacity Scheduler Intra-queue Preemption should be configurable for each queue	Major	capacity scheduler, scheduler preemption	Eric Payne	Eric Payne
HDFS-13175	Add more information for checking argument in DiskBalancerVolume	Minor	diskbalancer	Lei (Eddy) Xu	Lei (Eddy) Xu
HDFS-11187	Optimize disk access for last partial chunk checksum of Finalized replica	Major	datanode	Wei-Chiu Chuang	Gabor Bota
MAPREDUCE-7061	SingleCluster setup document needs to be updated	Major	.	Bharat Viswanadham	Bharat Viswanadham
HADOOP-15263	hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake	Minor	build	Steve Loughran	Steve Loughran
MAPREDUCE-7060	Cherry Pick PathOutputCommitter class/factory to branch-3.0	Minor	.	Steve Loughran	Steve Loughran
HADOOP-15279	increase maven heap size recommendations	Minor	build, documentation, test	Allen Wittenauer	Allen Wittenauer
HDFS-13170	Port webhdfs unmaskedpermission parameter to HTTPFS	Major	.	Stephen O'Donnell	Stephen O'Donnell
HDFS-13225	StripeReader#checkMissingBlocks() 's IOException info is incomplete	Major	erasure-coding, hdfs-client	lufei	lufei
HDFS-11394	Support for getting erasure coding policy through WebHDFS#FileStatus	Major	erasure-coding, namenode	Kai Sasaki	Kai Sasaki
HADOOP-15311	HttpServer2 needs a way to configure the acceptor/selector count	Major	common	Erik Krogen	Erik Krogen
HDFS-11600	Refactor TestDFSStripedOutputStreamWithFailure test classes	Minor	erasure-coding, test	Andrew Wang	Sammi Chen
HDFS-12884	BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo	Major	namenode	Konstantin Shvachko	chencan
HADOOP-15334	Upgrade Maven surefire plugin	Major	build	Arpit Agarwal	Arpit Agarwal
HADOOP-15312	Undocumented KeyProvider configuration keys	Major	.	Wei-Chiu Chuang	LiXin Ge
YARN-7623	Fix the CapacityScheduler Queue configuration documentation	Major	.	Arun Suresh	Jonathan Hung
HDFS-13314	NameNode should optionally exit if it detects FsImage corruption	Major	namenode	Arpit Agarwal	Arpit Agarwal
HADOOP-15342	Update ADLS connector to use the current SDK version (2.2.7)	Major	fs/adl	Atul Sikaria	Atul Sikaria
HDFS-13462	Add BIND_HOST configuration for JournalNode's HTTP and RPC Servers	Major	hdfs, journal-node	Lukas Majercak	Lukas Majercak
HADOOP-14841	Kms client should disconnect if unable to get output stream from connection.	Major	kms	Xiao Chen	Rushabh S Shah
HDFS-12981	renameSnapshot a Non-Existent snapshot to itself should throw error	Minor	hdfs	Sailesh Patel	Kitti Nanasi
YARN-8201	Skip stacktrace of few exception from ClientRMService	Minor	.	Bibin A Chundatt	Bilwa S T
HADOOP-15441	Log kms url and token service at debug level.	Minor	.	Wei-Chiu Chuang	Gabor Bota
HDFS-13544	Improve logging for JournalNode in federated cluster	Major	federation, hdfs	Hanisha Koneru	Hanisha Koneru
HADOOP-15486	Make NetworkTopology#netLock fair	Major	net	Nanda kumar	Nanda kumar
HDFS-13493	Reduce the HttpServer2 thread count on DataNodes	Major	datanode	Erik Krogen	Erik Krogen
HADOOP-15449	Increase default timeout of ZK session to avoid frequent NameNode failover	Critical	common	Karthik Palanisamy	Karthik Palanisamy

BUG FIXES:

JIRA	Summary	Priority	Component	Reporter	Contributor
HDFS-11968	ViewFS: StoragePolicies commands fail with HDFS federation	Major	hdfs	Mukul Kumar Singh	Mukul Kumar Singh
HDFS-12813	RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1	Major	ha	Mukul Kumar Singh	Mukul Kumar Singh
HDFS-13048	LowRedundancyReplicatedBlocks metric can be negative	Major	metrics	Akira Ajisaka	Akira Ajisaka
HDFS-13115	In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted	Major	.	Yongjun Zhang	Yongjun Zhang
HDFS-10453	ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.	Major	namenode	He Xiaoqiao	He Xiaoqiao
MAPREDUCE-7053	Timed out tasks can fail to produce thread dump	Major	.	Jason Lowe	Jason Lowe
HADOOP-15206	BZip2 drops and duplicates records when input split size is small	Major	.	Aki Tanaka	Aki Tanaka
YARN-7937	Fix http method name in Cluster Application Timeout Update API example request	Minor	docs, documentation	Charan Hebri	Charan Hebri
YARN-7947	Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps	Major	capacity scheduler, scheduler preemption	Eric Payne	Eric Payne
HADOOP-10571	Use Log.*(Object, Throwable) overload to log exceptions	Major	.	Arpit Agarwal	Andras Bokor
HDFS-12781	After Datanode down, In Namenode UI Datanode tab is throwing warning message.	Major	datanode	Harshakiran Reddy	Brahma Reddy Battula
HDFS-12070	Failed block recovery leaves files open indefinitely and at risk for data loss	Major	.	Daryn Sharp	Kihwal Lee
HDFS-13145	SBN crash when transition to ANN with in-progress edit tailing enabled	Major	ha, namenode	Chao Sun	Chao Sun
HDFS-13114	CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path	Major	encryption, hdfs	Hanisha Koneru	Hanisha Koneru
HDFS-13081	Datanode#checkSecureConfig should allow SASL and privileged HTTP	Major	datanode, security	Xiaoyu Yao	Ajay Kumar
MAPREDUCE-7059	Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster	Critical	job submission	Jiandan Yang	Jiandan Yang
HADOOP-15275	Incorrect javadoc for return type of RetryPolicy#shouldRetry	Minor	documentation	Nanda kumar	Nanda kumar
YARN-7511	NPE in ContainerLocalizer when localization failed for running container	Major	nodemanager	Tao Yang	Tao Yang
MAPREDUCE-7023	TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun	Minor	test	Gergely Novák	Gergely Novák
HDFS-13040	Kerberized inotify client fails despite kinit properly	Major	namenode	Wei-Chiu Chuang	Xiao Chen
YARN-7736	Fix itemization in YARN federation document	Minor	documentation	Akira Ajisaka	Sen Zhao
HDFS-13164	File not closed if streamer fail with DSQuotaExceededException	Major	hdfs-client	Xiao Chen	Xiao Chen
HADOOP-15289	FileStatus.readFields() assertion incorrect	Critical	.	Steve Loughran	Steve Loughran
HDFS-13109	Support fully qualified hdfs path in EZ commands	Major	hdfs	Hanisha Koneru	Hanisha Koneru
HADOOP-15296	Fix a wrong link for RBF in the top page	Minor	documentation	Takanobu Asanuma	Takanobu Asanuma
HADOOP-15273	distcp can't handle remote stores with different checksum algorithms	Critical	tools/distcp	Steve Loughran	Steve Loughran
MAPREDUCE-6930	mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml	Major	mrv2	Daniel Templeton	Sen Zhao
HDFS-13190	Document WebHDFS support for snapshot diff	Major	documentation, webhdfs	Xiaoyu Yao	Lokesh Jain
HDFS-13244	Add stack, conf, metrics links to utilities dropdown in NN webUI	Major	.	Bharat Viswanadham	Bharat Viswanadham
HDFS-12156	TestFSImage fails without -Pnative	Major	test	Akira Ajisaka	Akira Ajisaka
HDFS-13239	Fix non-empty dir warning message when setting default EC policy	Minor	.	Hanisha Koneru	Bharat Viswanadham
YARN-8022	ResourceManager UI cluster/app/<app-id> page fails to render	Blocker	webapp	Tarun Parimi	Tarun Parimi
MAPREDUCE-7064	Flaky test TestTaskAttempt#testReducerCustomResourceTypes	Major	client, test	Peter Bacsko	Peter Bacsko
HDFS-12723	TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently.	Major	.	Rushabh S Shah	Ajay Kumar
YARN-7636	Re-reservation count may overflow when cluster resource exhausted for a long time	Major	capacityscheduler	Tao Yang	Tao Yang
HDFS-12886	Ignore minReplication for block recovery	Major	hdfs, namenode	Lukas Majercak	Lukas Majercak
HDFS-13296	GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases	Major	.	Xiao Liang	Xiao Liang
HDFS-13268	TestWebHdfsFileContextMainOperations fails on Windows	Major	.	Íñigo Goiri	Xiao Liang
YARN-8054	Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread	Major	.	Jonathan Eagles	Jonathan Eagles
HDFS-13195	DataNode conf page cannot display the current value after reconfig	Minor	datanode	maobaolong	maobaolong
YARN-8063	DistributedShellTimelinePlugin wrongly check for entityId instead of entityType	Major	.	Rohith Sharma K S	Rohith Sharma K S
YARN-8062	yarn rmadmin -getGroups returns group from which the user has been removed	Critical	.	Sumana Sathish	Sunil Govindan
YARN-8068	Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+	Blocker	yarn	Sunil Govindan	Sunil Govindan
YARN-7734	YARN-5418 breaks TestContainerLogsPage.testContainerLogPageAccess	Major	.	Miklos Szegedi	Tao Yang
HDFS-13087	Snapshotted encryption zone information should be immutable	Major	encryption	LiXin Ge	LiXin Ge
HADOOP-12862	LDAP Group Mapping over SSL can not specify trust store	Major	.	Wei-Chiu Chuang	Wei-Chiu Chuang
HDFS-13349	Unresolved merge conflict in ViewFs.md	Blocker	documentation	Gera Shegalov	Yiqun Lin
HADOOP-15317	Improve NetworkTopology chooseRandom's loop	Major	.	Xiao Chen	Xiao Chen
HADOOP-15355	TestCommonConfigurationFields is broken by HADOOP-15312	Major	test	Konstantin Shvachko	LiXin Ge
HDFS-13350	Negative legacy block ID will confuse Erasure Coding to be considered as striped block	Major	erasure-coding	Lei (Eddy) Xu	Lei (Eddy) Xu
YARN-7905	Parent directory permission incorrect during public localization	Critical	.	Bibin A Chundatt	Bilwa S T
HDFS-13420	License header is displayed in ArchivalStorage/MemoryStorage html pages	Minor	documentation	Akira Ajisaka	Akira Ajisaka
HDFS-13328	Abstract ReencryptionHandler recursive logic in separate class.	Major	namenode	Surendra Singh Lilhore	Surendra Singh Lilhore
HADOOP-15357	Configuration.getPropsWithPrefix no longer does variable substitution	Major	.	Jim Brennan	Jim Brennan
MAPREDUCE-7062	Update mapreduce.job.tags description for making use for ATSv2 purpose.	Major	.	Charan Hebri	Charan Hebri
YARN-8073	TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration	Major	.	Rohith Sharma K S	Rohith Sharma K S
HDFS-13427	Fix the section titles of transparent encryption document	Minor	documentation	Akira Ajisaka	Akira Ajisaka
YARN-7527	Over-allocate node resource in async-scheduling mode of CapacityScheduler	Major	capacityscheduler	Tao Yang	Tao Yang
YARN-8120	JVM can crash with SIGSEGV when exiting due to custom leveldb logger	Major	nodemanager, resourcemanager	Jason Lowe	Jason Lowe
YARN-8147	TestClientRMService#testGetApplications sporadically fails	Major	test	Jason Lowe	Jason Lowe
HDFS-13436	Fix javadoc of package-info.java	Major	documentation	Akira Ajisaka	Akira Ajisaka
HADOOP-14970	MiniHadoopClusterManager doesn't respect lack of format option	Minor	.	Erik Krogen	Erik Krogen
HDFS-13330	ShortCircuitCache#fetchOrCreate never retries	Major	.	Wei-Chiu Chuang	Gabor Bota
YARN-8156	Increase the default value of yarn.timeline-service.app-collector.linger-period.ms	Major	.	Rohith Sharma K S	Charan Hebri
YARN-8165	Incorrect queue name logging in AbstractContainerAllocator	Trivial	capacityscheduler	Weiwei Yang	Weiwei Yang
HDFS-12828	OIV ReverseXML Processor fails with escaped characters	Critical	hdfs	Erik Krogen	Erik Krogen
HADOOP-15396	Some java source files are executable	Minor	.	Akira Ajisaka	Shashikant Banerjee
YARN-6827	[ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart.	Major	resourcemanager	Rohith Sharma K S	Rohith Sharma K S
YARN-7786	NullPointerException while launching ApplicationMaster	Major	.	lujie	lujie
HDFS-10183	Prevent race condition during class initialization	Minor	fs	Pavel Avgustinov	Pavel Avgustinov
HDFS-13388	RequestHedgingProxyProvider calls multiple configured NNs all the time	Major	hdfs-client	Jinglun	Jinglun
HDFS-13433	webhdfs requests can be routed incorrectly in federated cluster	Critical	webhdfs	Arpit Agarwal	Arpit Agarwal
HDFS-13408	MiniDFSCluster to support being built on randomized base directory	Major	test	Xiao Liang	Xiao Liang
HADOOP-15390	Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens	Critical	.	Xiao Chen	Xiao Chen
HDFS-13336	Test cases of TestWriteToReplica failed in windows	Major	.	Xiao Liang	Xiao Liang
YARN-7598	Document how to use classpath isolation for aux-services in YARN	Major	.	Xuan Gong	Xuan Gong
YARN-8183	Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap	Critical	yarn	Sumana Sathish	Suma Shivaprasad
HADOOP-15411	AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator	Critical	.	Suma Shivaprasad	Suma Shivaprasad
MAPREDUCE-7042	Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled	Major	.	Yesha Vora	Xuan Gong
YARN-8205	Application State is not updated to ATS if AM launching is delayed.	Critical	.	Sumana Sathish	Rohith Sharma K S
YARN-8004	Add unit tests for inter queue preemption for dominant resource calculator	Critical	yarn	Sumana Sathish	Zian Chen
YARN-8221	RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps	Major	webapp	Sunil Govindan	Sunil Govindan
YARN-8210	AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs	Major	yarn	Suma Shivaprasad	Suma Shivaprasad
HDFS-13509	Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows	Major	.	Xiao Liang	Xiao Liang
MAPREDUCE-7073	Optimize TokenCache#obtainTokensForNamenodesInternal	Major	.	Bibin A Chundatt	Bibin A Chundatt
HADOOP-15406	hadoop-nfs dependencies for mockito and junit are not test scope	Major	nfs	Jason Lowe	Jason Lowe
YARN-6385	Fix checkstyle warnings in TestFileSystemApplicationHistoryStore	Minor	.	Yiqun Lin	Yiqun Lin
YARN-8222	Fix potential NPE when gets RMApp from RM context	Critical	.	Tao Yang	Tao Yang
HDFS-13481	TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently	Major	hdfs	Gabor Bota	Gabor Bota
YARN-8217	RmAuthenticationFilterInitializer /TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator	Major	.	Suma Shivaprasad	Suma Shivaprasad
YARN-8025	UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty	Major	yarn	Jiandan Yang	Tao Yang
YARN-8232	RMContainer lost queue name when RM HA happens	Major	resourcemanager	Hu Ziqian	Hu Ziqian
HDFS-13136	Avoid taking FSN lock while doing group member lookup for FSD permission check	Major	namenode	Xiaoyu Yao	Xiaoyu Yao
HDFS-13537	TestHdfsHelper does not generate jceks path properly for relative path in Windows	Major	.	Xiao Liang	Xiao Liang
YARN-7003	DRAINING state of queues is not recovered after RM restart	Major	capacityscheduler	Tao Yang	Tao Yang
YARN-8244	TestContainerSchedulerQueuing.testStartMultipleContainers failed	Major	.	Miklos Szegedi	Jim Brennan
YARN-8288	Fix wrong number of table columns in Resource Model doc	Major	.	Weiwei Yang	Weiwei Yang
HDFS-13539	DFSStripedInputStream NPE when reportCheckSumFailure	Major	.	Xiao Chen	Xiao Chen
YARN-8278	DistributedScheduling is not working in HA	Blocker	.	Bibin A Chundatt	Bibin A Chundatt
HDFS-13581	DN UI logs link is broken when https is enabled	Minor	datanode	Namit Maheshwari	Shashikant Banerjee
HDFS-13586	Fsync fails on directories on Windows	Critical	datanode, hdfs	Lukas Majercak	Lukas Majercak
YARN-8179	Preemption does not happen due to natural_termination_factor when DRF is used	Major	.	kyungwan nam	kyungwan nam
HADOOP-15450	Avoid fsync storm triggered by DiskChecker and handle disk full situation	Blocker	.	Kihwal Lee	Arpit Agarwal
HDFS-13601	Optimize ByteString conversions in PBHelper	Major	.	Andrew Wang	Andrew Wang
HDFS-13540	DFSStripedInputStream should only allocate new buffers when reading	Major	.	Xiao Chen	Xiao Chen
HDFS-13588	Fix TestFsDatasetImpl test failures on Windows	Major	.	Xiao Liang	Xiao Liang
YARN-8310	Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats	Major	.	Robert Kanter	Robert Kanter
YARN-8344	Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync	Major	.	Giovanni Matteo Fumarola	Giovanni Matteo Fumarola
YARN-8327	Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows	Major	log-aggregation	Giovanni Matteo Fumarola	Giovanni Matteo Fumarola
YARN-8346	Upgrading to 3.1 kills running containers with error "Opportunistic container queue is full"	Blocker	.	Rohith Sharma K S	Jason Lowe
HDFS-13611	Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient	Major	.	Andrew Wang	Andrew Wang
HDFS-13618	Fix TestDataNodeFaultInjector test failures on Windows	Major	test	Xiao Liang	Xiao Liang
HADOOP-15473	Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997	Critical	kms	Gabor Bota	Gabor Bota
YARN-8338	TimelineService V1.5 doesn't come up after HADOOP-15406	Critical	.	Vinod Kumar Vavilapalli	Vinod Kumar Vavilapalli
HADOOP-9747	Reduce unnecessary UGI synchronization	Critical	security	Daryn Sharp	Daryn Sharp
YARN-8518	test-container-executor test_is_empty() is broken	Major	.	Jim Brennan	Jim Brennan

TESTS:

JIRA	Summary	Priority	Component	Reporter	Contributor
HADOOP-15313	TestKMS should close providers	Major	kms, test	Xiao Chen	Xiao Chen
HDFS-13503	Fix TestFsck test failures on Windows	Major	hdfs	Xiao Liang	Xiao Liang
HDFS-13542	TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13551	TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster	Minor	.	Anbang Hu	Anbang Hu
HDFS-11700	TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13548	TestResolveHdfsSymlink#testFcResolveAfs fails on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13567	TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir	Minor	.	Anbang Hu	Anbang Hu
HDFS-13557	TestDFSAdmin#testListOpenFiles fails on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13550	TestDebugAdmin#testComputeMetaCommand fails on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13559	TestBlockScanner does not close TestContext properly	Minor	.	Anbang Hu	Anbang Hu
HDFS-13570	TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13558	TestDatanodeHttpXFrame does not shut down cluster	Minor	.	Anbang Hu	Anbang Hu
HDFS-13554	TestDatanodeRegistration#testForcedRegistration does not shut down cluster	Minor	.	Anbang Hu	Anbang Hu
HDFS-13556	TestNestedEncryptionZones does not shut down cluster	Minor	.	Anbang Hu	Anbang Hu
HDFS-13560	Insufficient system resources exist to complete the requested service for some tests on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13592	TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly	Minor	.	Anbang Hu	Anbang Hu
HDFS-13593	TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows	Minor	test	Anbang Hu	Anbang Hu
HDFS-13587	TestQuorumJournalManager fails on Windows	Minor	.	Anbang Hu	Anbang Hu
HDFS-13619	TestAuditLoggerWithCommands fails on Windows	Minor	test	Anbang Hu	Anbang Hu
HDFS-13620	Randomize the test directory path for TestHDFSFileSystemContract	Minor	.	Anbang Hu	Anbang Hu
HDFS-13591	TestDFSShell#testSetrepLow fails on Windows	Minor	.	Anbang Hu	Anbang Hu

SUB-TASKS:

JIRA	Summary	Priority	Component	Reporter	Contributor
HDFS-13068	RBF: Add router admin option to manage safe mode	Major	.	Íñigo Goiri	Yiqun Lin
HADOOP-15040	Upgrade AWS SDK to 1.11.271: NPE bug spams logs w/ Yarn Log Aggregation	Blocker	fs/s3	Aaron Fabbri	Aaron Fabbri
HDFS-13119	RBF: Manage unavailable clusters	Major	.	Íñigo Goiri	Yiqun Lin
HADOOP-15247	Move commons-net up to 3.6	Minor	fs	Steve Loughran	Steve Loughran
HDFS-13187	RBF: Fix Routers information shown in the web UI	Minor	.	Wei Yan	Wei Yan
HDFS-13184	RBF: Improve the unit test TestRouterRPCClientRetries	Minor	test	Yiqun Lin	Yiqun Lin
HDFS-13199	RBF: Fix the hdfs router page missing label icon issue	Major	federation, hdfs	maobaolong	maobaolong
HADOOP-15264	AWS "shaded" SDK 1.11.271 is pulling in netty 4.1.17	Blocker	fs/s3	Steve Loughran	Steve Loughran
HADOOP-15090	Add ADL troubleshooting doc	Major	documentation, fs/adl	Steve Loughran	Steve Loughran
HDFS-13214	RBF: Complete document of Router configuration	Major	.	Tao Jie	Yiqun Lin
HADOOP-15267	S3A multipart upload fails when SSE-C encryption is enabled	Critical	fs/s3	Anis Elleuch	Anis Elleuch
HDFS-13230	RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns	Minor	.	Wei Yan	Chao Sun
HDFS-13233	RBF: MountTableResolver doesn't return the correct mount point of the given path	Major	hdfs	wangzhiyuan	wangzhiyuan
HADOOP-15277	remove .FluentPropertyBeanIntrospector from CLI operation log output	Minor	conf	Steve Loughran	Steve Loughran
HDFS-13212	RBF: Fix router location cache issue	Major	federation, hdfs	Weiwei Wu	Weiwei Wu
HDFS-13232	RBF: ConnectionPool should return first usable connection	Minor	.	Wei Yan	Ekanth Sethuramalingam
HDFS-13240	RBF: Update some inaccurate document descriptions	Minor	.	Yiqun Lin	Yiqun Lin
HDFS-11399	Many tests fails in Windows due to injecting disk failures	Major	.	Yiqun Lin	Yiqun Lin
HDFS-13241	RBF: TestRouterSafemode failed if the port 8888 is in use	Major	hdfs, test	maobaolong	maobaolong
HDFS-13253	RBF: Quota management incorrect parent-child relationship judgement	Major	.	Yiqun Lin	Yiqun Lin
HDFS-13226	RBF: Throw the exception if mount table entry validated failed	Major	hdfs	maobaolong	maobaolong
HDFS-12505	Extend TestFileStatusWithECPolicy with a random EC policy	Major	erasure-coding, test	Takanobu Asanuma	Takanobu Asanuma
HDFS-12587	Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies	Major	erasure-coding, test	Takanobu Asanuma	Takanobu Asanuma
HADOOP-15308	TestConfiguration fails on Windows because of paths	Major	test	Íñigo Goiri	Xiao Liang
HDFS-12773	RBF: Improve State Store FS implementation	Major	.	Íñigo Goiri	Íñigo Goiri
HDFS-13198	RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router	Minor	.	Wei Yan	Wei Yan
HDFS-13224	RBF: Resolvers to support mount points across multiple subclusters	Major	.	Íñigo Goiri	Íñigo Goiri
HADOOP-15262	AliyunOSS: move files under a directory in parallel when rename a directory	Major	fs/oss	wujinhu	wujinhu
HDFS-13215	RBF: Move Router to its own module	Major	.	Íñigo Goiri	Wei Yan
HDFS-13250	RBF: Router to manage requests across multiple subclusters	Major	.	Íñigo Goiri	Íñigo Goiri
HDFS-13318	RBF: Fix FindBugs in hadoop-hdfs-rbf	Minor	.	Íñigo Goiri	Ekanth Sethuramalingam
HDFS-12792	RBF: Test Router-based federation using HDFSContract	Major	.	Íñigo Goiri	Íñigo Goiri
YARN-7986	ATSv2 REST API queries do not return results for uppercase application tags	Critical	.	Charan Hebri	Charan Hebri
HDFS-12512	RBF: Add WebHDFS	Major	fs	Íñigo Goiri	Wei Yan
HDFS-13291	RBF: Implement available space based OrderResolver	Major	.	Yiqun Lin	Yiqun Lin
HDFS-13204	RBF: Optimize name service safe mode icon	Minor	.	liuhongtong	liuhongtong
HDFS-13352	RBF: Add xsl stylesheet for hdfs-rbf-default.xml	Major	documentation	Takanobu Asanuma	Takanobu Asanuma
HDFS-13347	RBF: Cache datanode reports	Minor	.	Íñigo Goiri	Íñigo Goiri
HDFS-13289	RBF: TestConnectionManager#testCleanup() test case need correction	Minor	.	Dibyendu Karmakar	Dibyendu Karmakar
HDFS-13364	RBF: Support NamenodeProtocol in the Router	Major	.	Íñigo Goiri	Íñigo Goiri
HADOOP-14651	Update okhttp version to 2.7.5	Major	fs/adl	Ray Chiang	Ray Chiang
YARN-6936	[Atsv2] Retrospect storing entities into sub application table from client perspective	Major	.	Rohith Sharma K S	Rohith Sharma K S
HDFS-13353	RBF: TestRouterWebHDFSContractCreate failed	Major	test	Takanobu Asanuma	Takanobu Asanuma
YARN-8107	Give an informative message when incorrect format is used in ATSv2 filter attributes	Major	ATSv2	Charan Hebri	Rohith Sharma K S
HDFS-13402	RBF: Fix java doc for StateStoreFileSystemImpl	Minor	hdfs	Yiran Wu	Yiran Wu
HDFS-13410	RBF: Support federation with no subclusters	Minor	.	Íñigo Goiri	Íñigo Goiri
HDFS-13384	RBF: Improve timeout RPC call mechanism	Minor	.	Íñigo Goiri	Íñigo Goiri
HDFS-13045	RBF: Improve error message returned from subcluster	Minor	.	Wei Yan	Íñigo Goiri
HDFS-13428	RBF: Remove LinkedList From StateStoreFileImpl.java	Trivial	federation	BELUGA BEHR	BELUGA BEHR
HDFS-13386	RBF: Wrong date information in list file(-ls) result	Minor	.	Dibyendu Karmakar	Dibyendu Karmakar
YARN-8027	Setting hostname of docker container breaks for --net=host in docker 1.13	Major	yarn	Jim Brennan	Jim Brennan
HADOOP-14999	AliyunOSS: provide one asynchronous multi-part based uploading mechanism	Major	fs/oss	Genmao Yu	Genmao Yu
YARN-7810	TestDockerContainerRuntime test failures due to UID lookup of a non-existent user	Major	.	Shane Kumpf	Shane Kumpf
HDFS-13435	RBF: Improve the error loggings for printing the stack trace	Major	.	Yiqun Lin	Yiqun Lin
YARN-7189	Container-executor doesn't remove Docker containers that error out early	Major	yarn	Eric Badger	Eric Badger
HDFS-13466	RBF: Add more router-related information to the UI	Minor	.	Wei Yan	Wei Yan
HDFS-13453	RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table	Major	.	Dibyendu Karmakar	Dibyendu Karmakar
HDFS-13478	RBF: Disabled Nameservice store API	Major	.	Íñigo Goiri	Íñigo Goiri
HDFS-13490	RBF: Fix setSafeMode in the Router	Major	.	Íñigo Goiri	Íñigo Goiri
HDFS-13484	RBF: Disable Nameservices from the federation	Major	.	Íñigo Goiri	Íñigo Goiri
HDFS-13326	RBF: Improve the interfaces to modify and view mount tables	Minor	.	Wei Yan	Gang Li
HDFS-13499	RBF: Show disabled name services in the UI	Minor	.	Íñigo Goiri	Íñigo Goiri
YARN-8215	ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM	Critical	ATSv2	Yesha Vora	Rohith Sharma K S
HDFS-13508	RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries	Minor	.	Ekanth Sethuramalingam	Ekanth Sethuramalingam
HDFS-13434	RBF: Fix dead links in RBF document	Major	documentation	Akira Ajisaka	Chetna Chaudhari
YARN-8212	Pending backlog for async allocation threads should be configurable	Major	.	Weiwei Yang	Tao Yang
HDFS-13488	RBF: Reject requests when a Router is overloaded	Major	.	Íñigo Goiri	Íñigo Goiri
HDFS-13525	RBF: Add unit test TestStateStoreDisabledNameservice	Major	.	Yiqun Lin	Yiqun Lin
YARN-8253	HTTPS Ats v2 api call fails with "bad HTTP parsed"	Critical	ATSv2	Yesha Vora	Charan Hebri
HADOOP-15454	TestRollingFileSystemSinkWithLocal fails on Windows	Major	test	Xiao Liang	Xiao Liang
YARN-8247	Incorrect HTTP status code returned by ATSv2 for non-whitelisted users	Critical	ATSv2	Charan Hebri	Rohith Sharma K S
YARN-8130	Race condition when container events are published for KILLED applications	Major	ATSv2	Charan Hebri	Rohith Sharma K S
HADOOP-15498	TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows	Minor	.	Anbang Hu	Anbang Hu
HADOOP-15497	TestTrash should use proper test path to avoid failing on Windows	Minor	.	Anbang Hu	Anbang Hu
YARN-4781	Support intra-queue preemption for fairness ordering policy.	Major	scheduler	Wangda Tan	Eric Payne

OTHER:

JIRA	Summary	Priority	Component	Reporter	Contributor
HDFS-13052	WebHDFS: Add support for snasphot diff	Major	.	Lokesh Jain	Lokesh Jain
HADOOP-14742	Document multi-URI replication Inode for ViewFS	Major	documentation, viewfs	Chris Douglas	Gera Shegalov