Back to Hadoop

Licensed to the Apache Software Foundation (ASF) under one

hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/CHANGELOG.0.16.0.md

2.0.5-alpha-rc232.0 KB
Original Source
<!--- # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -->

Apache Hadoop Changelog

Release 0.16.0 - 2008-02-07

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-2045credits page should have more informationMajordocumentationDoug CuttingDoug Cutting
HADOOP-1604admins should be able to finalize namenode upgrades without running the clusterCritical.Owen O'MalleyKonstantin Shvachko
HADOOP-1912Datanode should support block replacementMajor.Hairong KuangHairong Kuang
HADOOP-2288Change FileSystem API to support access control.MajorfsTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-2229Provide a simple login implementationMajorfsTsz Wo Nicholas SzeHairong Kuang
HADOOP-2184RPC Support for user permissions and authentication.MajoripcTsz Wo Nicholas SzeRaghu Angadi
HADOOP-1652Rebalance data blocks when new data nodes added or data nodes become fullMajor.Hairong KuangHairong Kuang
HADOOP-2145need 'doc' target that runs forrestMajorbuildDoug CuttingDoug Cutting
HADOOP-2085Map-side joins on sorted, equally-partitioned datasetsMajor.Chris DouglasChris Douglas
HADOOP-2381Support permission information in FileStatusMajorfsTsz Wo Nicholas SzeRaghu Angadi
HADOOP-2336Shell commands to access and modify file permissionsMajorfsRaghu AngadiRaghu Angadi
HADOOP-1301resource management proviosioning for HadoopMajor.Pete WyckoffHemanth Yamijala
HADOOP-1298adding user info to fileMajorfsKurtis HeimerlTsz Wo Nicholas Sze
HADOOP-2447HDFS should be capable of limiting the total number of inodes in the systemMajor.Sameer Paranjpyedhruba borthakur
HADOOP-2487Provide an option to get job status for all jobs run by or submitted to a job trackerMajor.Hemanth YamijalaAmareshwari Sriramadasu
HADOOP-2398Additional Instrumentation for NameNode, RPC Layer and JMX supportMajor.Sanjay RadiaSanjay Radia
HADOOP-2514Trash and permissions don't mixMajor.Robert ChanslerDoug Cutting
HADOOP-2012Periodic verification at the DatanodeMajor.Raghu AngadiRaghu Angadi
HADOOP-2543No-permission-checking mode for smooth transition to 0.16's permissions features.Major.Sanjay RadiaHairong Kuang
HADOOP-2603SequenceFileAsBinaryInputFormatMajor.Chris DouglasChris Douglas
HADOOP-2367Get representative hprof information from tasksMajor.Owen O'MalleyOwen O'Malley
HADOOP-2567add FileSystem#getHomeDirectory() methodMajorfsDoug CuttingDoug Cutting
HADOOP-2529DFS User GuideMajordocumentationRaghu AngadiRaghu Angadi

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-2058Allow adding additional datanodes to MiniDFSClusterMajortestHairong KuangHairong Kuang
HADOOP-1855fsck should verify block placementMajor.dhruba borthakurKonstantin Shvachko
HADOOP-1839Link-ify the Pending/Running/Complete/Killed tasks/task-attempts on jobdetails.jspMajor.Arun C MurthyAmar Kamat
HADOOP-1848Redesign of Eclipse plug-in interface with HadoopMajor.Christophe TatonChristophe Taton
HADOOP-1857Ability to run a script when a task fails to capture stack tracesMajor.Amareshwari SriramadasuAmareshwari Sriramadasu
HADOOP-1210Log counters in job historyMinor.Albert ChernOwen O'Malley
HADOOP-2105Clarify requirements for Hadoop in overview.htmlMinor.Jim KellermanJim Kellerman
HADOOP-2086ability to add dependencies to a job after constructionMajor.Adrian WoodheadAdrian Woodhead
HADOOP-1185dynamically change log levelsMajorutildhruba borthakurTsz Wo Nicholas Sze
HADOOP-2134Remove developer-centric requirements from overview.htmlMajordocumentationArun C MurthyJim Kellerman
HADOOP-1274Configuring different number of mappers and reducers per TaskTrackerMajor.Koji NoguchiAmareshwari Sriramadasu
HADOOP-2127Add pipes sort exampleMajor.Owen O'MalleyOwen O'Malley
HADOOP-1843Remove deprecated code in Configuration/JobConfMajorconfArun C MurthyArun C Murthy
HADOOP-2113Add "-text" command to FsShell to decode SequenceFile to stdoutMinorfsChris DouglasChris Douglas
HADOOP-1900the heartbeat and task event queries interval should be set dynamically by the JobTrackerMajor.Owen O'MalleyAmareshwari Sriramadasu
HADOOP-1302Remove deprecated contrib/abacus codeMajor.Doug CuttingEnis Soztutar
HADOOP-2349FSEditLog.logEdit(byte op, Writable w1, Writable w2) should accept variable numbers of Writable, instead of two.Minor.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-2185Server ports: to roll or not to roll.MajorconfKonstantin ShvachkoKonstantin Shvachko
HADOOP-2401Lease holder information should be passed in ClientProtocol.abandonBlock(...)Major.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-1841IPC server should write repsonses asynchronouslyMajoripcDoug Cuttingdhruba borthakur
HADOOP-2432If HDFS is going to throw an exception "File does not exist" it should include the name of the fileMinor.Jim KellermanJim Kellerman
HADOOP-2457Add a 'forrest.home' property for the 'docs' target in build.xmlMinordocumentationArun C MurthyArun C Murthy
HADOOP-2149Pure name-node benchmarks.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-2466FileInputFormat computeSplitSize() method, change visibility to protected and make it a member methodMajor.Alejandro AbdelnurAlejandro Abdelnur
HADOOP-2425TextOutputFormat should special case TextMajor.Owen O'MalleyOwen O'Malley
HADOOP-2208Reduce frequency of Counter updates in the task tracker statusMajor.Amareshwari SriramadasuAmareshwari Sriramadasu
HADOOP-1719Improve the utilization of shuffle copier threadsMajor.Devaraj DasAmar Kamat
HADOOP-2390Document the user-controls for intermediate/output compression via forrestMajordocumentationArun C MurthyArun C Murthy
HADOOP-1660add support for native library toDistributedCacheMajor.Alejandro AbdelnurArun C Murthy
HADOOP-2233General example for modeling m/r load in JavaMinortestChris DouglasChris Douglas
HADOOP-2547remove use of 'magic number' in build.xmlTrivialbuildHrishikeshHrishikesh
HADOOP-2268JobControl classes should use interfaces rather than implemenationsMinor.Adrian WoodheadAdrian Woodhead
HADOOP-2552enable hdfs permission checking by defaultMajor.Doug CuttingDoug Cutting
HADOOP-2131Speculative execution should be allowed for reducers onlyCritical.Srikanth KakaniAmareshwari Sriramadasu
HADOOP-1873User permissions for Map/ReduceMajor.Raghu AngadiHairong Kuang
HADOOP-1965Handle map output buffers betterMajor.Devaraj DasAmar Kamat
HADOOP-2464Test permissions related shell commands with DFSMajor.Raghu AngadiRaghu Angadi
HADOOP-1876Persisting completed jobs statusCritical.Alejandro AbdelnurAlejandro Abdelnur
HADOOP-2077Logging version number (and compiled date) at STARTUP_MSGTrivial.Koji NoguchiArun C Murthy
HADOOP-1989Add support for simulated Data Nodes - helpful for testing and performance benchmarking of the Name Node without having a large clusterMinor.Sanjay RadiaSanjay Radia
HADOOP-1742FSNamesystem.startFile() javadoc is inconsistentMinor.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1707Remove the DFS Client disk-based cacheMajor.dhruba borthakurdhruba borthakur
HADOOP-2298ant target without source and docsMajorbuildGautam KowshikHrishikesh
HADOOP-2469WritableUtils.clone should take Configuration rather than JobConfMinoriostackstack
HADOOP-2596add SequenceFile.createWriter() method that takes block size as parameterMinorioAlejandro AbdelnurAlejandro Abdelnur
HADOOP-2645Additional metrics & jmx beans and cleanup to use the recent metrics librariesMajor.Sanjay RadiaSanjay Radia
HADOOP-2232Add option to disable nagles algorithm in the IPC ServerMajoripcClint MorganClint Morgan
HADOOP-2566need FileSystem#globStatus methodMajorfsDoug CuttingHairong Kuang

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-2072RawLocalFileStatus is causing Path problemsMajorfsDennis Kubes
HADOOP-1245value for mapred.tasktracker.tasks.maximum taken from jobtracker, not tasktrackerMajor.Michael BieniosekMichael Bieniosek
HADOOP-1898locking for the ReflectionUtils.logThreadInfo is too conservativeMajorutilOwen O'MalleyAmareshwari Sriramadasu
HADOOP-2100hadoop-daemon.sh script fails if HADOOP_PID_DIR doesn't existMajorscriptsArun C MurthyMichael Bieniosek
HADOOP-2096The file used to localize job.xml should be closed.Minor.Amar KamatAmar Kamat
HADOOP-2098File handles for log files are still open in case of jobs with 0 mapsMajor.Amar KamatAmar Kamat
HADOOP-2112TestMiniMRMapRedDebugScript fails due to a missing fileBlocker.Devaraj DasArun C Murthy
HADOOP-2089Multiple caheArchive does not work in Hadoop streamingCritical.Milind BhandarkarLohit Vijayarenu
HADOOP-2071StreamXmlRecordReader throws java.io.IOException: Mark/reset exception in hadoop 0.14Major.Lohit VijayarenuLohit Vijayarenu
HADOOP-1348Configuration XML bug: comments inside valuesCriticalconfEelco LempsinkRajagopal Natarajan
HADOOP-1952Streaming does not handle invalid -inputformat (typo by users for example)Minor.Lohit VijayarenuLohit Vijayarenu
HADOOP-2151FileSyste.globPaths does not validate the return list of PathsMajor.Lohit VijayarenuLohit Vijayarenu
HADOOP-2121Unexpected IOException in DFSOutputStream.close()Major.Raghu AngadiRaghu Angadi
HADOOP-1130Remove unused ClientFinalizer in DFSClientMajor.Philippe GassmannChris Douglas
HADOOP-2204DFSTestUtil.waitReplication does not wait.Major.Raghu AngadiRaghu Angadi
HADOOP-2078Name-node should be able to close empty files.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1642Jobs using LocalJobRunner + JobControl failsCritical.Johan OskarssonDoug Cutting
HADOOP-2104clover description attribute suppresses all other targets in -projecthelpTrivialbuildChris DouglasChris Douglas
HADOOP-2212java.lang.ArithmeticException: / by zero in ChecksumFileSystem.openCriticalfsMichael BieniosekMichael Bieniosek
HADOOP-2216Job UI doesnot show running tasks and complete tasks correctly.Major.Amareshwari SriramadasuAmar Kamat
HADOOP-2272findbugs currently fails due to hadoop-streaming having movedMajorbuildAdrian Woodheadstack
HADOOP-2244MapWritable.readFields needs to clear internal hash else instance accumulates entries foreverMajoriostackstack
HADOOP-1984some reducer stuck at copy phase and progress extremely slowlyCritical.Runping QiAmar Kamat
HADOOP-2245TestRecordMR and TestAggregates fail once in a whileMajor.Devaraj DasAdrian Woodhead
HADOOP-2275Erroneous detection of corrupted file when namenode fails to allocate any datanodes for newly allocated blockMajor.dhruba borthakurdhruba borthakur
HADOOP-2256TestBlockReplacement unit test failed.Major.Raghu AngadiHairong Kuang
HADOOP-2209SecondaryNamenode process should exit if it encounters Runtime exceptionsMajor.dhruba borthakur
HADOOP-2314TestBlockReplacement occasionally get into an infinite loopMajor.Hairong KuangHairong Kuang
HADOOP-2300mapred.tasktracker.tasks.maximum is completely ignoredBlocker.Owen O'MalleyAmareshwari Sriramadasu
HADOOP-2368Lots of unit tests fail on Windows with exception: Login failed: CreateProcessBlocker.Mukund MadhugiriTsz Wo Nicholas Sze
HADOOP-2363Unit tests fail if there is another instance of HadoopMajortestRaghu AngadiKonstantin Shvachko
HADOOP-2271chmod in ant package target failsMajorbuildAdrian WoodheadAdrian Woodhead
HADOOP-2313build does not fail when libhdfs build failsMinor.Nigel DaleyNigel Daley
HADOOP-2359PendingReplicationMonitor thread received exception. java.lang.InterruptedExceptionMajor.Owen O'Malleydhruba borthakur
HADOOP-2365Result of HashFunction.hash() contains all identical valuesMinor.Andrzej BialeckiJim Kellerman
HADOOP-2323JobTracker.close() prints stack traces for exceptions that are not errorsMinor.Jim KellermanJim Kellerman
HADOOP-2376The sort example shouldn't override the number of mapsMajor.Owen O'MalleyOwen O'Malley
HADOOP-2248Word count example is spending 24% of the time in incrCounterMajor.Owen O'MalleyOwen O'Malley
HADOOP-2228Jobs fail because job.xml existsMajor.Johan OskarssonJohan Oskarsson
HADOOP-2434MapFile.get on HDFS in TRUNK is WAY!!! slower than 0.15.xBlockeriostackstack
HADOOP-2459Running 'ant docs tar' includes src/docs/build in the resulting tar fileMinorbuildNigel DaleyNigel Daley
HADOOP-2453wordcount-simple example gives ParseException with examples configuration fileBlocker.Amareshwari SriramadasuAmareshwari Sriramadasu
HADOOP-2215Change documentation in cluster_setup.html and mapred_tutorial.html post HADOOP-1274MajordocumentationAmareshwari SriramadasuAmareshwari Sriramadasu
HADOOP-2352Remove AC_LIB_CHECK from src/native/configure.ac to ensure libhadoop.so doesn't have a dependency on libz.so/liblzo.soMajornativeArun C MurthyArun C Murthy
HADOOP-2220Reduce tasks fail too easily because of repeated fetch failuresBlocker.Christian KunzAmar Kamat
HADOOP-2247Mappers fail easily due to repeated failuresBlocker.Srikanth KakaniAmar Kamat
HADOOP-2452Eclipse plug-in build.xml issueTrivialbuildChristophe TatonChristophe Taton
HADOOP-2476Unit test fails on Windows: TestCopyFiles.testCopyFromLocalToLocalBlockerfsMukund MadhugiriRaghu Angadi
HADOOP-2503REST Insert / SelectCritical.Billy PearsonBryan Duxbury
HADOOP-2492ConcurrentModificationException in org.apache.hadoop.ipc.Server.ResponderMajoripcDevaraj Dasdhruba borthakur
HADOOP-2344Free up the buffers (input and error) while executing a shell command before waiting for it to finish.Major.Amar KamatAmar Kamat
HADOOP-2511HADOOP-2344 introduced a javadoc warningMajordocumentationArun C MurthyArun C Murthy
HADOOP-2442Unit test failed: org.apache.hadoop.fs.TestLocalFileSystemPermission.testLocalFSsetOwnerCriticalfsMukund MadhugiriRaghu Angadi
HADOOP-2523Unit test fails on Windows: TestDFSShell.testFilePermissionsBlocker.Mukund MadhugiriRaghu Angadi
HADOOP-2535Remove support for deprecated mapred.child.heap.size and indentation fix in TaskRunner.javaMinor.Arun C MurthyArun C Murthy
HADOOP-2512error stream handling in Shell executorBlockerutilRaghu AngadiRaghu Angadi
HADOOP-2420Use exit code to detect normal errors while excuting 'ls' in Local FSBlockerfsRaghu AngadiRaghu Angadi
HADOOP-2285TextInputFormat is slow compared to reading files.Major.Owen O'MalleyOwen O'Malley
HADOOP-2446TestHDFSServerPorts fails.MajortestRaghu AngadiNigel Daley
HADOOP-2537make build process compatible with Ant 1.7.0MajorbuildNigel DaleyHrishikesh
HADOOP-1281Speculative map tasks aren't getting killed although the TIP completedCritical.Arun C MurthyArun C Murthy
HADOOP-2571javac generates a warning in test/o.a.h.io.FileBenchTrivialtestChris DouglasChris Douglas
HADOOP-2583Potential Eclipse plug-in UI loop when editing location parametersMinorcontrib/eclipse-pluginChristophe TatonChristophe Taton
HADOOP-2481NNBench should periodically report its progressMajortestHairong KuangHairong Kuang
HADOOP-2601TestNNThroughput should not use a fixed namenode portMajor.Hairong KuangKonstantin Shvachko
HADOOP-2494Set +x on contrib/*/bin/* in packaged tar bundleMajorscriptsstackstack
HADOOP-2605leading slash in mapred.task.tracker.report.bindAddressMajorconfKonstantin ShvachkoKonstantin Shvachko
HADOOP-2620'bin/hadoop fs -help' does not list file permissions commands.TrivialfsRaghu AngadiRaghu Angadi
HADOOP-2614dfs web interfaces should run as a configurable user accountMajor.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-290Fix Datanode transfer thread loggingMinor.Dennis Kubesdhruba borthakur
HADOOP-2538NPE in TaskLog.javaTrivial.Michael BieniosekMichael Bieniosek
HADOOP-2582hadoop dfs -copyToLocal creates zero byte files, when source file does not existsMajor.Lohit VijayarenuLohit Vijayarenu
HADOOP-2189Incrementing user counters should count as progressBlocker.Owen O'MalleyDevaraj Das
HADOOP-2284BasicTypeSorterBase.compare calls progress on each compareMajor.Owen O'MalleyAmar Kamat
HADOOP-2649The ReplicationMonitor sleep period should be configurableBlocker.dhruba borthakurdhruba borthakur
HADOOP-2549hdfs does not honor dfs.du.reserved settingCritical.Joydeep Sen SarmaHairong Kuang
HADOOP-2509Add rat target to buildMajorbuildNigel DaleyHrishikesh
HADOOP-2659The commands in DFSAdmin should require admin privilegeMajor.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-2633Revert change to fsck made as part of permissions implementationBlocker.Robert ChanslerTsz Wo Nicholas Sze
HADOOP-26871707 added errant INFO-level logging to DFSClientBlocker.stackstack
HADOOP-2402Lzo compression compresses each write from TextOutputFormatMajorio, nativeChris DouglasChris Douglas
HADOOP-2691Some junit tests fail with the exception: All datanodes are bad. Aborting...Major.Hairong Kuangdhruba borthakur
HADOOP-1195NullPointerException in FSNamesystem due to getDatanode() return value is not checkedMajor.Konstantin Shvachkodhruba borthakur
HADOOP-2640MultiFileSplitInputFormat always returns 1 split when avgLengthPerSplit > Integer.MAX_VALUEBlocker.Frédéric BertinEnis Soztutar
HADOOP-2626RawLocalFileStatus is badly handling URIsMajorfsFrédéric BertinDoug Cutting
HADOOP-2646SortValidator broken with fully-qualified working directoriesBlockertestDoug CuttingArun C Murthy
HADOOP-2652Fix permission issues for HftpFileSystemBlocker.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-2092Pipes C++ task does not die even if the Java tasks dieBlocker.Amareshwari SriramadasuAmareshwari Sriramadasu
HADOOP-2714Unit test fails on Windows: rg.apache.hadoop.dfs.TestDecommissionBlocker.Mukund Madhugiridhruba borthakur
HADOOP-2576Namenode performance degradation over timeBlocker.Christian KunzRaghu Angadi
HADOOP-2720Update HOD in Hadoop 0.16Blockercontrib/hodHemanth YamijalaHemanth Yamijala
HADOOP-2713Unit test fails on Windows: org.apache.hadoop.dfs.TestDatanodeDeathBlocker.Mukund Madhugiridhruba borthakur
HADOOP-2639Reducers stuck in shuffleBlocker.Amareshwari SriramadasuArun C Murthy
HADOOP-2723Hadoop 2367- Does not respect JobConf.getProfileEnabled()Blocker.Clint MorganAmareshwari Sriramadasu
HADOOP-2734docs link to lucene.apache.orgMajordocumentationDoug CuttingDoug Cutting
HADOOP-2732ab{5[6-9],[6-9][6-9]}.gz should not be treated as an illegal globBlockerfsHairong KuangHairong Kuang
HADOOP-2404HADOOP-2185 breaks compatibility with hadoop-0.15.0BlockerconfArun C MurthyOwen O'Malley
HADOOP-2740Modify HOD to work with changes mentioned in HADOOP-2404Blockercontrib/hodHemanth YamijalaHemanth Yamijala
HADOOP-2755dfs fsck extremely slow, dfs ls times outBlocker.Christian KunzTsz Wo Nicholas Sze
HADOOP-2768DFSIO write performance benchmark shows a regressionBlocker.Mukund Madhugiridhruba borthakur

TESTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-2000Re-write NNBench to use MapReduceMajortestMukund MadhugiriMukund Madhugiri
HADOOP-2369Representative mix of jobs for large cluster throughput benchmarkingMajortestChris DouglasRunping Qi
HADOOP-2406Micro-benchmark to measure read/write times through InputFormatsMajorfs, testChris DouglasChris Douglas
HADOOP-2449Restore the old NN Bench that was replaced by a MR NN BenchMajor.Sanjay RadiaSanjay Radia
HADOOP-2431Test HDFS File PermissionsMajortestHairong KuangHairong Kuang
HADOOP-2342create a micro-benchmark for measure local-file versus hdfs readMajor.Owen O'MalleyOwen O'Malley