docs/en/contributor/Code-Conventions.md
First off, we thank you for your interest in the Alluxio open source project! We greatly appreciate any contribution; whether it be new features or bug fixes.
If you are a first time contributor to the Alluxio open source project, we strongly encourage you to follow the step-by-step instructions within the [Contribution Guide]({{ '/en/contributor/Contributor-Getting-Started.html' | relativize_url }}) and finish new contributor tasks before making more advanced changes to the Alluxio codebase.
Submitting changes to Alluxio is done via pull requests. Please read our [pull request guidelines]({{ '/en/contributor/Contributor-Getting-Started.html' | relativize_url }}#sending-a-pull-request) for details on how to submit a pull request to the Alluxio repository. Below are some tips for the pull requests.
Fixes #1234.Please follow the style of the existing codebase. We mainly follow the Google Java style, with the following deviations:
m
private WorkerClient mWorkerClient;s
private static String sUnderFSAddress;Bash scripts follow the Google Shell style, and must be compatible with Bash 3.x
If you use IntelliJ IDEA:
If you use Eclipse:
To automatically reorder methods alphabetically, try the Rearranger Plugin, open Preferences, search for rearranger, remove the unnecessary comments, then right click, choose "Rearrange", codes will be formatted to what you want
To verify that the coding standards match, you should run checkstyle before sending a pull request to verify no new warnings are introduced:
$ mvn checkstyle:checkstyle
This codebase follows the Oracle JavaDoc style with the following refinements:
All public classes/interfaces should have a class/interface-level comment that describes the purpose of the class/interface.
All public members should have a member-level comment the describes the purpose of the member.
/** The number of logical bytes used. */
public final AtomicLong mBytes = new AtomicLong(0);
/**
* Does something. This is a method description that uses
* 3rd person (does something) as opposed to 2nd person (do
* something).
*
* @param param_1 description of 1st parameter
* ...
* @param param_n description of nth parameter
* @return description of return argument (if applicable)
* @throws exception_1 description of 1st exception case
* ...
* @throws exception_n description of nth exception case
*/
An exception to the above rule is that @throws doesn’t need to be provided for @Test methods,
or for generic exceptions like IOException when there is nothing interesting to document.
Only write exception javadoc when you think it will be useful to the developer using the method.
There are so many sources of IOException that it’s almost never useful to include javadoc for it.
Do not write javadoc for unchecked exceptions like RuntimeException unless it is critical for this method.
Getters and setters should omit the method description if it is redundant and only use @param and @return descriptions.
/**
* @return the number of pages stored
*/
long getPages();
Most sentences should start with a capital letter and end with a period. An exception to this style is an isolated sentence; it does not start with a capital letter nor end with a period.
When writing the description, the first sentence should be a concise summary of the class or method and the description should generally be implementation-independent. It is a good idea to use additional sentences to describe any significant performance implications.
/**
* The default implementation of a metadata store for pages stored in cache.
*/
public class DefaultMetaStore implements MetaStore {
...
}
@deprecated annotation is added, it should also at least tell the user when the API was deprecated and what to use as a replacement with @see or @link tag./**
* @deprecated as of Alluxio 2.1, replaced by
* {@link #newMethodName(int,int,int,int)}
*/
@param, @return, @throw exceed one line,
the text should align with the first argument after the tag.@throws FileAlreadyExistsException if there is already a file or directory at the given path
in Alluxio Filesystem
<code>ClassName</code> tags to {@link ClassName}.Alluxio uses SLF4J for logging with typical usage pattern of:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public MyClass {
private static final Logger LOG = LoggerFactory.getLogger(MyClass.class);
public void someMethod() {
LOG.info("Hello world");
}
}
Note that, each class must use its own logger based on the class name,
like LoggerFactory.getLogger(MyClass.class) in above example,
so its output can easily be searched for.
The location of the output of SLF4J loggers can be found for
[server logs]({{ '/en/administration/Basic-Logging.html' | relativize_url }}#server-logs)
and [application logs]({{ '/en/administration/Basic-Logging.html' | relativize_url }}#application-logs).
// Recommended: Parameterized logging
LOG.debug("Client {} registered with {}", mClient, mHostname);
// Not recommended: Non-parameterized logging, hard to read and expensive due to String
// concatenation regardless of if DEBUG is enabled
LOG.debug("Client " + mClient + " registered with " + mHostname);
// Recommended: error messages with context
LOG.error("Client {} failed to register to {}", mClient, mHostname, exception);
// Not recommended: There is no information on which client failed, or what the issue was
LOG.error("Client failed to register");
INFO and above should be easily human readablevariable: valuetoString() implementationsThere are several levels of logging, see detailed explanation of [different Levels]({{ '/en/administration/Basic-Logging.html' | relativize_url }}#configuring-log-levels) Here are the guidelines for deciding which level to use.
Error level logging (LOG.error) indicates system level problems which cannot be recovered from.
It should be accompanied by a stack trace of the exception thrown to help debug the issue.
// Recommended: a stack trace will be shown in the log
LOG.error("Failed to do something due to an exception", e);
// Not recommended: wrong logging syntax
LOG.error("Failed to do something due to an exception {}", e);
// Not recommended: stack trace will not be logged
LOG.error("Failed to do something due to an exception {}", e.toString());
When to Use
During Exception Handling
Warn level logging (LOG.warn) indicates a logical mismatch between user intended behavior
and Alluxio behavior.
Warn level logs are accompanied by an exception message using e.toString().
The associated stack trace is typically not included to avoid spamming the logs.
If needed, print the details in separate debug level logs.
// Recommended
LOG.warn("Failed to do something: {}", e.toString());
// Not recommended: wrong logging syntax
LOG.warn("Failed to do something: {}", e);
// Not recommended: this will print out the stack trace, can be noisy
LOG.warn("Failed to do something", e);
// Not recommended: the exception class name is not included
LOG.warn("Failed to do something {}", e.getMessage());
When to Use
During Exception Handling
Info level logging (LOG.info) records important system state changes. Exception messages
and stack traces are never associated with info level logs. Note that, this level of logging should
not be used on critical path of operations that may happen frequently to prevent negative performance
impact.
LOG.info("Master started with address: {}.", address);
When to Use
During Exception Handling
Debug level logging (LOG.debug) includes detailed information for various aspects of
the Alluxio system. Control flow logging (Alluxio system enter and exit calls) is done in debug
level logs. Debug level logging of exceptions typically has the detailed information including
stack trace. Please avoid the slow strings construction on debug-level logging on critical path.
// Recommended
LOG.debug("Failed to connect to {} due to exception", mAddress, e);
// Recommended: string concatenation is only performed if debug is enabled
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to connect to address {} due to exception", host + ":" + port, e);
}
// Not recommended: string concatenation is always performed
LOG.debug("Failed to connect to {} due to exception", host + ":" + port, e);
When to Use
During Exception Handling
When to Use
During Exception Handling
These are the guidelines for throwing and handling exceptions throughout the Alluxio codebase.
These are the guidelines for how and when to throw exceptions.
Examples: Illegal states, invalid API usage
public void method(String neverSupposedToBeNull) {
Preconditions.checkNotNull(neverSupposedToBeNull, "neverSupposedToBeNull");
}
Examples: File not found, interrupted exception, timeout exceeded
// Recommended: the checked exception should just be propagated
public void handleRawUserInput(String date) throws InvalidDateException {
parseDate(date);
}
// Don't do this - it's reasonable for user input to be invalid.
public void handleRawUserInput(String date) throws InvalidDateException {
try {
parseDate(date);
} catch (InvalidDateException e) {
throw new RuntimeExcepiton("date " + date + " is invalid", e);
}
}
Require callers to validate inputs so that invalid arguments can be unchecked exceptions.
Try to find an appropriate existing exception before inventing a new one.
Only write exception javadoc when you think it will be useful to the developer using the method. There are so many sources of IOException that it's almost never useful to include javadoc for it.
On the wire we represent exceptions with one of 14 status codes, e.g. NOT_FOUND, UNAVAILABLE.
Within our server and client code, we represent these exceptions using exception classes
corresponding to these statuses, e.g. NotFoundException and UnavailableException.
AlluxioStatusException is the superclass for these Java exceptions.
These are the guidelines for how to handle exceptions.
Either log the exception or propagate it.
It is usually wrong to both log and propagate. If every method did this, the same exception would be logged dozens of times, polluting the logs. The responsibility for logging lies with whatever method ends up handling the exception without propagating it.
See this explanation on why Throwables.propagate is deprecated.
An InterruptedException means that another thread has signalled that this thread should die. There are
a few acceptable ways to handle an InterruptedException, listed in order of preference.
Actually stop the thread
class MyRunnable implements Runnable {
public void run() {
while (true) {
try {
doSomethingThatThrowsInterruptedException();
} catch (InterruptedException e) {
break;
}
}
}
}
Directly propagate the exception
public void myMethod() throws InterruptedException {
doSomethingThatThrowsInterruptedException();
}
Reset the interrupted flag and continue
This punts the problem forward. Hopefully later code will inspect the interrupted flag and handle it appropriately.
public void myMethod() {
try {
doSomethingThatThrowsInterruptedException();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
Reset the interrupted flag and wrap the InterruptedException
public void myMethod() {
try {
doSomethingThatThrowsInterruptedException();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException(e);
}
}
Assume that any method might throw an unchecked exception and make sure this doesn't cause resource
leaks. We do not stop servers when RPC threads throw RuntimeExceptions. try-finally and
try-with-resources blocks can make releasing resources much easier.
// with try-finally
Resource r = acquireResource();
try {
doSomething(r);
} finally {
r.close();
}
// with try-with-resources
try (Resource r = acquireResource()) {
doSomething(r);
}
If both the try block and the finally block throw exceptions, the exception from the try block will be lost and the finally block exception will be thrown. This is almost never desirable since the exception in the try block happened first. To avoid this, either make sure your finally blocks can't throw exceptions, or use Guava's Closer.
Guava's Closer is helpful for reducing boilerplate and making exception handling less error-prone, for releasing resources.
Closer closer = new Closer();
closer.register(resource1);
closer.register(resource2);
closer.close();
If both calls to close() throw an exception, the first exception will be thrown and the second
exception will be added as a suppressed exception of the first one.
From the Closer javadoc:
Closer closer = Closer.create();
try {
InputStream in = closer.register(openInputStream());
OutputStream out = closer.register(openOutputStream());
// do stuff
} catch (Throwable e) {
// ensure that any checked exception types other than IOException that could be thrown are
// provided here, e.g. throw closer.rethrow(e, CheckedException.class);
throw closer.rethrow(e);
} finally {
closer.close();
}
mCloser = new Closer();
mCloser.register(new resource1());
try {
doSomething();
} catch (Throwable t) {
// We want to close resources and throw the original exception t. Any exceptions thrown while
// closing resources should be suppressed on t.
try {
throw mCloser.rethrow(t);
} finally {
mCloser.close();
}
}
We have a util method for closing a closer and suppressing its exceptions onto an existing exception
mCloser = new Closer();
mCloser.register(new resource1());
try {
doSomething();
} catch (Throwable t) {
throw CommonUtils.closeAndRethrow(mCloser, t);
}
This will improve static analysis of our code so that we can detect potential NullPointerExceptions
before they happen.
Use the javax.annotation.Nullable import.
import javax.annotation.Nullable;
@Nullable
public String getName() {
if (mName == "") {
return null;
}
return mName;
}
When a method is specifically designed to be able to handle null parameters, those parameters
should be annotated with @Nullable.
Use the javax.annotation.Nullable import.
import javax.annotation.Nullable;
public repeat(@Nullable String s) {
if (s == null) {
System.out.println("Hello world");
} else {
System.out.println(s);
}
}
The preconditions check gives a more useful error message when you tell it the name of the variable being checked.
Preconditions.checkNotNull(blockInfo, "blockInfo") // Do this
Preconditions.checkNotNull(blockInfo); // Do NOT do this
Tests are easier to read when there is less boilerplate. Use static imports for methods in
org.junit.Assert, org.junit.Assume, org.mockito.Matchers, and org.mockito.Mockito.
// Change
Assert.assertFalse(fileInfo.isFolder());
// to
assertFalse(fileInfo.isFolder());
// It may be necessary to add the import
import static org.junit.Assert.assertFalse;
@Before method to perform shared setup steps.
The @Before method gets run automatically before each unit test.
Test-specific setup should be done locally in the tests that need it.
In this example, we are testing a BlockMaster, which depends on a journal, clock, and executor service.
The executor service and journal we provide are real implementations,
and the TestClock is a fake clock which can be controlled by unit tests.@Before
public void before() throws Exception {
Journal blockJournal = new ReadWriteJournal(mTestFolder.newFolder().getAbsolutePath());
mClock = new TestClock();
mExecutorService =
Executors.newFixedThreadPool(2, ThreadFactoryUtils.build("TestBlockMaster-%d", true));
mMaster = new BlockMaster(blockJournal, mClock, mExecutorService);
mMaster.start(true);
}
@Before creates something which needs to be cleaned up (e.g. a
BlockMaster), create an @After method to do the cleanup.
This method is automatically called after each test.@After
public void after() throws Exception {
mMaster.stop();
}
Decide on an element of functionality to test. The functionality you decide to test should be part of the public API and should not care about implementation details. Tests should be focused on testing only one thing.
Give your test a name that describes what functionality it's testing. The functionality being
tested should ideally be simple enough to fit into a name, e.g.
removeNonexistentBlockThrowsException, mkdirCreatesDirectory, or cannotMkdirExistingFile.
@Test
public void detectLostWorker() throws Exception {
HeartbeatScheduler section enforces that the lost worker heartbeat runs at least
once.// Register a worker.
long worker1 = mMaster.getWorkerId(NET_ADDRESS_1);
mMaster.workerRegister(worker1,
ImmutableList.of("MEM"),
ImmutableMap.of("MEM", 100L),
ImmutableMap.of("MEM", 10L),
NO_BLOCKS_ON_TIERS);
// Advance the block master's clock by an hour so that the worker appears lost.
mClock.setTimeMs(System.currentTimeMillis() + Constants.HOUR_MS);
// Run the lost worker detector.
HeartbeatScheduler.await(HeartbeatContext.MASTER_LOST_WORKER_DETECTION, 1, TimeUnit.SECONDS);
HeartbeatScheduler.schedule(HeartbeatContext.MASTER_LOST_WORKER_DETECTION);
HeartbeatScheduler.await(HeartbeatContext.MASTER_LOST_WORKER_DETECTION, 1, TimeUnit.SECONDS);
// Make sure the worker is detected as lost.
Set<WorkerInfo> info = mMaster.getLostWorkersInfo();
assertEquals(worker1, Iterables.getOnlyElement(info).getId());
}
src/main/java/ClassName.java should go in src/test/java/ClassNameTest.javathrows Exception to the test method signature.Thread.sleep(). This leads to slower unit tests and can
cause flaky failures if the sleep isn't long enough.All tests in a module run in the same JVM, so it's important to properly manage global state so
that tests don't interfere with each other. Global state includes system properties, Alluxio
configuration, and any static fields. Our solution to managing global state is to use JUnit's
support for @Rules.
Some unit tests want to test Alluxio under different configurations. This requires modifying the
global Configuration object. When all tests in a suite need configuration parameters set a
certain way, use ConfigurationRule to set them.
@Rule
public ConfigurationRule mConfigurationRule = new ConfigurationRule(ImmutableMap.of(
PropertyKey.key1, "value1",
PropertyKey.key2, "value2"));
For configuration changes needed for an individual test, use ConfigurationRule#set(key, value) on the instance created by your test class. The resulting change of this method is only visible for the calling test.
@Rule
public ConfigurationRule mConfigurationRule = new ConfigurationRule(ImmutableMap.of(
PropertyKey.key1, "value1",
PropertyKey.key2, "value2"));
@Test
public void testSomething() {
mConfigurationRule.set(PropertyKey.key1, "value3");
// Now PropertyKey.key1 = "value3"
...
}
@Test
public void testAnotherThing() {
// Now PropertyKey.key1 = "value1"
}
If you need to change a system property for the duration of a test suite, use SystemPropertyRule.
@Rule
public SystemPropertyRule mSystemPropertyRule = new SystemPropertyRule("propertyName", "value");
To set a system property during a specific test, use the SystemPropertyRule#toResource() method
to get a Closeable for a try-catch statement:
@Test
public void test() {
try (Closeable p = new SystemPropertyRule("propertyKey", "propertyValue").toResource()) {
// Test something with propertyKey set to propertyValue.
}
}
If a test needs to modify other types of global state, create a new @Rule for managing the
state so that it can be shared across tests.
One example of this is
TtlIntervalRule.
Sometimes you will need to play with a few system settings in order to have the unit tests pass
locally. A common setting that may need to be set is ulimit.
In order to increase the number of files and processes allowed on MacOS, run the following
$ sudo launchctl limit maxfiles 32768 32768
$ sudo launchctl limit maxproc 32768 32768
It is also recommended to exclude your local clone of Alluxio from Spotlight indexing. Otherwise,
your Mac may hang constantly trying to re-index the file system during the unit tests. To do this,
go to System Preferences > Spotlight > Privacy, click the + button, browse to the directory
containing your local clone of Alluxio, and click Choose to add it to the exclusions list.