doc/releases/infernalis.rst
.. _infernalis-release-notes:
Infernalis is the 9th stable release of Ceph. It is named after the vampire squid (Vampyroteuthis infernalis, the "vampire squid from hell").
This Infernalis point release fixes several packagins and init script issues, enables the librbd objectmap feature by default, a few librbd bugs, and a range of miscellaneous bug fixes across the system.
We recommend that all infernalis v9.2.0 users upgrade.
For more detailed information, see :download:the complete changelog <../changelog/v9.2.1.txt>.
issue#13474 <http://tracker.ceph.com/issues/13474>, pr#6833 <http://github.com/ceph/ceph/pull/6833>, Chuanhong Wang)issue#13214 <http://tracker.ceph.com/issues/13214>, pr#6396 <http://github.com/ceph/ceph/pull/6396>, Sage Weil)issue#13997 <http://tracker.ceph.com/issues/13997>, pr#6845 <http://github.com/ceph/ceph/pull/6845>, Loic Dachary)issue#13785 <http://tracker.ceph.com/issues/13785>, pr#6628 <http://github.com/ceph/ceph/pull/6628>, Loic Dachary)issue#13975 <http://tracker.ceph.com/issues/13975>, pr#6852 <http://github.com/ceph/ceph/pull/6852>, Loic Dachary)issue#13374 <http://tracker.ceph.com/issues/13374>, pr#6500 <http://github.com/ceph/ceph/pull/6500>, Boris Ranto)issue#13610 <http://tracker.ceph.com/issues/13610>, pr#6836 <http://github.com/ceph/ceph/pull/6836>, Qiankun Zheng)issue#13993 <http://tracker.ceph.com/issues/13993>, pr#6993 <http://github.com/ceph/ceph/pull/6993>, Brad Hubbard)issue#13477 <http://tracker.ceph.com/issues/13477>, pr#6626 <http://github.com/ceph/ceph/pull/6626>, Sage Weil)issue#13777 <http://tracker.ceph.com/issues/13777>, pr#7431 <http://github.com/ceph/ceph/pull/7431>, Yan, Zheng, John Spray)issue#13443 <http://tracker.ceph.com/issues/13443>, pr#6395 <http://github.com/ceph/ceph/pull/6395>, Wenjun Huang)issue#13714 <http://tracker.ceph.com/issues/13714>, pr#6853 <http://github.com/ceph/ceph/pull/6853>, Yan, Zheng)issue#13674 <http://tracker.ceph.com/issues/13674>, pr#6630 <http://github.com/ceph/ceph/pull/6630>, Jason Dillaman)issue#14373 <http://tracker.ceph.com/issues/14373>, pr#7406 <http://github.com/ceph/ceph/pull/7406>, Jason Dillaman)issue#13726 <http://tracker.ceph.com/issues/13726>, pr#6632 <http://github.com/ceph/ceph/pull/6632>, Jason Dillaman)issue#13633 <http://tracker.ceph.com/issues/13633>, pr#7080 <http://github.com/ceph/ceph/pull/7080>, Jason Dillaman)issue#14164 <http://tracker.ceph.com/issues/14164>, pr#7079 <http://github.com/ceph/ceph/pull/7079>, Jason Dillaman)issue#13558 <http://tracker.ceph.com/issues/13558>, pr#6477 <http://github.com/ceph/ceph/pull/6477>, xinxin shu)issue#14030 <http://tracker.ceph.com/issues/14030>, pr#6981 <http://github.com/ceph/ceph/pull/6981>, Jason Dillaman)issue#14092 <http://tracker.ceph.com/issues/14092>, issue#14483 <http://tracker.ceph.com/issues/14483>, pr#7484 <http://github.com/ceph/ceph/pull/7484>_, Jason Dillaman)issue#11482 <http://tracker.ceph.com/issues/11482>, pr#6752 <http://github.com/ceph/ceph/pull/6752>, Yan, Zheng)issue#13840 <http://tracker.ceph.com/issues/13840>, pr#6907 <http://github.com/ceph/ceph/pull/6907>, Chengyuan Li)issue#13525 <http://tracker.ceph.com/issues/13525>, pr#6392 <http://github.com/ceph/ceph/pull/6392>, Ruifeng Yang)issue#13642 <http://tracker.ceph.com/issues/13642>, pr#6627 <http://github.com/ceph/ceph/pull/6627>, xie xingguo)issue#13639 <http://tracker.ceph.com/issues/13639>, pr#6840 <http://github.com/ceph/ceph/pull/6840>, xie xingguo)issue#13597 <http://tracker.ceph.com/issues/13597>, pr#6846 <http://github.com/ceph/ceph/pull/6846>, xie xingguo)issue#13471 <http://tracker.ceph.com/issues/13471>, pr#6397 <http://github.com/ceph/ceph/pull/6397>, xiexingguo)issue#13962 <http://tracker.ceph.com/issues/13962>, pr#6849 <http://github.com/ceph/ceph/pull/6849>, Sage Weil)issue#13970 <http://tracker.ceph.com/issues/13970>, issue#14230 <http://tracker.ceph.com/issues/14230>, pr#6880 <http://github.com/ceph/ceph/pull/6880>_, Loic Dachary)issue#14080 <http://tracker.ceph.com/issues/14080>, pr#7001 <http://github.com/ceph/ceph/pull/7001>, Loic Dachary, Ilya Dryomov)issue#13821 <http://tracker.ceph.com/issues/13821>, pr#6694 <http://github.com/ceph/ceph/pull/6694>, Sage Weil)issue#14015 <http://tracker.ceph.com/issues/14015>, pr#6851 <http://github.com/ceph/ceph/pull/6851>, David Zafman)issue#13862 <http://tracker.ceph.com/issues/13862>, pr#7421 <http://github.com/ceph/ceph/pull/7421>, David Zafman)issue#14476 <http://tracker.ceph.com/issues/14476>, pr#7426 <http://github.com/ceph/ceph/pull/7426>, Douglas Fuller)issue#14280 <http://tracker.ceph.com/issues/14280>, pr#7428 <http://github.com/ceph/ceph/pull/7428>, Douglas Fuller)issue#13491 <http://tracker.ceph.com/issues/13491>, pr#6629 <http://github.com/ceph/ceph/pull/6629>, Jason Dillaman)issue#13636 <http://tracker.ceph.com/issues/13636>, pr#6633 <http://github.com/ceph/ceph/pull/6633>, Jason Dillaman)issue#13553 <http://tracker.ceph.com/issues/13553>, pr#6474 <http://github.com/ceph/ceph/pull/6474>, xinxin shu)issue#14452 <http://tracker.ceph.com/issues/14452>, pr#7427 <http://github.com/ceph/ceph/pull/7427>, Mykola Golub)issue#13560 <http://tracker.ceph.com/issues/13560>, pr#6634 <http://github.com/ceph/ceph/pull/6634>, Jason Dillaman)issue#13709 <http://tracker.ceph.com/issues/13709>, pr#6650 <http://github.com/ceph/ceph/pull/6650>, Hervé Rousseau)issue#14516 <http://tracker.ceph.com/issues/14516>, pr#7543 <http://github.com/ceph/ceph/pull/7543>, Yehuda Sadeh)issue#14004 <http://tracker.ceph.com/issues/14004>, pr#6828 <http://github.com/ceph/ceph/pull/6828>, Loic Dachary)issue#14384 <http://tracker.ceph.com/issues/14384>, pr#7429 <http://github.com/ceph/ceph/pull/7429>, Greg Farnum)issue#13810 <http://tracker.ceph.com/issues/13810>, pr#7225 <http://github.com/ceph/ceph/pull/7225>, Jason Dillaman)issue#14121 <http://tracker.ceph.com/issues/14121>, pr#7038 <http://github.com/ceph/ceph/pull/7038>, Jason Dillaman)issue#13664 <http://tracker.ceph.com/issues/13664>, pr#6449 <http://github.com/ceph/ceph/pull/6449>, Loic Dachary)issue#13988 <http://tracker.ceph.com/issues/13988>, pr#6882 <http://github.com/ceph/ceph/pull/6882>, Loic Dachary)This major release will be the foundation for the next stable series. There have been some major changes since v0.94.x Hammer, and the upgrade process is non-trivial. Please read these release notes carefully.
General:
RADOS:
RGW:
RBD:
rbd du command shows actual usage (quickly, when
object-map is enabled).--size 64G).rbd status command that, for now, shows who has
the image open/mapped.CephFS:
We have decided to drop support for many older distributions so that we can move to a newer compiler toolchain (e.g., C++11). Although it is still possible to build Ceph on older distributions by installing backported development tools, we are not building and publishing release packages for ceph.com.
We now build packages for:
Upgrading directly from Firefly v0.80.z is not recommended. It is possible to do a direct upgrade, but not without downtime. We recommend that clusters are first upgraded to Hammer v0.94.4 or a later v0.94.z release; only then is it possible to upgrade to Infernalis 9.2.z for an online upgrade (see below).
To do an offline upgrade directly from Firefly, all Firefly OSDs must be stopped and marked down before any Infernalis OSDs will be allowed to start up. This fencing is enforced by the Infernalis monitor, so use an upgrade procedure like:
#. Upgrade Ceph on monitor hosts #. Restart all ceph-mon daemons #. Upgrade Ceph on all OSD hosts #. Stop all ceph-osd daemons #. Mark all OSDs down with something like::
ceph osd down `seq 0 1000`
#. Start all ceph-osd daemons #. Upgrade and restart remaining daemons (ceph-mds, radosgw)
All cluster nodes must first upgrade to Hammer v0.94.4 or a later v0.94.z release; only then is it possible to upgrade to Infernalis 9.2.z.
For all distributions that support systemd (CentOS 7, Fedora, Debian Jessie 8.x, OpenSUSE), ceph daemons are now managed using native systemd files instead of the legacy sysvinit scripts. For example::
systemctl start ceph.target # start all daemons systemctl status ceph-osd@12 # check status of osd.12
The main notable distro that is not yet using systemd is Ubuntu trusty 14.04. (The next Ubuntu LTS, 16.04, will use systemd instead of upstart.)
Ceph daemons now run as user and group ceph by default. The
ceph user has a static UID assigned by Fedora and Debian (also used
by derivative distributions like RHEL/CentOS and Ubuntu). On SUSE
the ceph user will currently get a dynamically assigned UID when the
user is created.
If your systems already have a ceph user, upgrading the package will cause problems. We suggest you first remove or rename the existing 'ceph' user and 'ceph' group before upgrading.
When upgrading, administrators have two options:
#. Add the following line to ceph.conf on all hosts::
setuser match path = /var/lib/ceph/$type/$cluster-$id
This will make the Ceph daemons run as root (i.e., not drop
privileges and switch to user ceph) if the daemon's data
directory is still owned by root. Newly deployed daemons will
be created with data owned by user ceph and will run with
reduced privileges, but upgraded daemons will continue to run as
root.
#. Fix the data ownership during the upgrade. This is the preferred option, but it is more work and can be very time consuming. The process for each host is to:
#. Upgrade the ceph package. This creates the ceph user and group. For example::
ceph-deploy install --stable infernalis HOST
#. Stop the daemon(s)::
service ceph stop # fedora, centos, rhel, debian
stop ceph-all # ubuntu
#. Fix the ownership::
chown -R ceph:ceph /var/lib/ceph
chown -R ceph:ceph /var/log/ceph
#. Restart the daemon(s)::
start ceph-all # ubuntu
systemctl start ceph.target # debian, centos, fedora, rhel
Alternatively, the same process can be done with a single daemon
type, for example by stopping only monitors and chowning only
``/var/lib/ceph/mon``.
The on-disk format for the experimental KeyValueStore OSD backend has changed. You will need to remove any OSDs using that backend before you upgrade any test clusters that use it.
When a pool quota is reached, librados operations now block indefinitely, the same way they do when the cluster fills up. (Previously they would return -ENOSPC). By default, a full cluster or pool will now block. If your librados application can handle ENOSPC or EDQUOT errors gracefully, you can get error returns instead by using the new librados OPERATION_FULL_TRY flag.
The return code for librbd's rbd_aio_read and Image::aio_read API methods no longer returns the number of bytes read upon success. Instead, it returns 0 upon success and a negative value upon failure.
'ceph scrub', 'ceph compact' and 'ceph sync force are now DEPRECATED. Users should instead use 'ceph mon scrub', 'ceph mon compact' and 'ceph mon sync force'.
'ceph mon_metadata' should now be used as 'ceph mon metadata'. There is no need to deprecate this command (same major release since it was first introduced).
The --dump-json option of "osdmaptool" is replaced by --dump json.
The commands of "pg ls-by-{pool,primary,osd}" and "pg ls" now take "recovering" instead of "recovery", to include the recovering pgs in the listed pgs.
This is the first Infernalis release candidate. There have been some major changes since Hammer, and the upgrade process is non-trivial. Please read carefully.
The v9.1.0 packages are pushed to the development release repositories::
http://download.ceph.com/rpm-testing http://download.ceph.com/debian-testing
For for info, see::
http://docs.ceph.com/docs/master/install/get-packages/
Or install with ceph-deploy via::
ceph-deploy install --testing HOST
General:
RADOS:
RGW:
RBD:
rbd du command shows actual usage (quickly, when
object-map is enabled).--size 64G).rbd status command that, for now, shows who has
the image open/mapped.CephFS:
We have decided to drop support for many older distributions so that we can move to a newer compiler toolchain (e.g., C++11). Although it is still possible to build Ceph on older distributions by installing backported development tools, we are not building and publishing release packages for them on ceph.com.
In particular,
Upgrading directly from Firefly v0.80.z is not possible. All clusters must first upgrade to Hammer v0.94.4 or a later v0.94.z release; only then is it possible to do online upgrade to Infernalis 9.2.z.
User can upgrade to latest hammer v0.94.z from gitbuilder with(also refer the hammer release notes for more details)::
ceph-deploy install --release hammer HOST
All cluster nodes must first upgrade to Hammer v0.94.4 or a later v0.94.z release; only then is it possible to do online upgrade to Infernalis 9.2.z.
For all distributions that support systemd (CentOS 7, Fedora, Debian Jessie 8.x, OpenSUSE), ceph daemons are now managed using native systemd files instead of the legacy sysvinit scripts. For example::
systemctl start ceph.target # start all daemons systemctl status ceph-osd@12 # check status of osd.12
The main notable distro that is not yet using systemd is Ubuntu trusty 14.04. (The next Ubuntu LTS, 16.04, will use systemd instead of upstart.)
Ceph daemons now run as user and group ceph by default. The
ceph user has a static UID assigned by Fedora and Debian (also used
by derivative distributions like RHEL/CentOS and Ubuntu). On SUSE
the ceph user will currently get a dynamically assigned UID when the
user is created.
If your systems already have a ceph user, the package upgrade process will usually fail with an error. We suggest you first remove or rename the existing 'ceph' user and then upgrade.
When upgrading, administrators have two options:
#. Add the following line to ceph.conf on all hosts::
setuser match path = /var/lib/ceph/$type/$cluster-$id
This will make the Ceph daemons run as root (i.e., not drop
privileges and switch to user ceph) if the daemon's data
directory is still owned by root. Newly deployed daemons will
be created with data owned by user ceph and will run with
reduced privileges, but upgraded daemons will continue to run as
root.
#. Fix the data ownership during the upgrade. This is the preferred option, but is more work. The process for each host would be to:
#. Upgrade the ceph package. This creates the ceph user and group. For
example::
ceph-deploy install --stable infernalis HOST
#. Stop the daemon(s)::
service ceph stop # fedora, centos, rhel, debian
stop ceph-all # ubuntu
#. Fix the ownership::
chown -R ceph:ceph /var/lib/ceph
chown -R ceph:ceph /var/log/ceph
#. Restart the daemon(s)::
start ceph-all # ubuntu
systemctl start ceph.target # debian, centos, fedora, rhel
The on-disk format for the experimental KeyValueStore OSD backend has changed. You will need to remove any OSDs using that backend before you upgrade any test clusters that use it.
NOTE: These notes are somewhat abbreviated while we find a less time-consuming process for generating them.
This is the second to last batch of development work for the Infernalis cycle. The most intrusive change is an internal (non user-visible) change to the OSD's ObjectStore interface. Many fixes and improvements elsewhere across RGW, RBD, and another big pile of CephFS scrub/repair improvements.
The return code for librbd's rbd_aio_read and Image::aio_read API methods no longer returns the number of bytes read upon success. Instead, it returns 0 upon success and a negative value upon failure.
'ceph scrub', 'ceph compact' and 'ceph sync force' are now deprecated. Users should instead use 'ceph mon scrub', 'ceph mon compact' and 'ceph mon sync force'.
'ceph mon_metadata' should now be used as 'ceph mon metadata'.
The --dump-json option of "osdmaptool" is replaced by --dump json.
The commands of 'pg ls-by-{pool,primary,osd}' and 'pg ls' now take 'recovering' instead of 'recovery' to include the recovering pgs in the listed pgs.
This development release features more of the OSD work queue unification, randomized osd scrub times, a huge pile of librbd fixes, more MDS repair and snapshot fixes, and a significant amount of work on the tests and build infrastructure.
This development release is delayed a bit due to tooling changes in the build environment. As a result the next one (v9.0.2) will have a bit more work than is usual.
Highlights here include lots of RGW Swift fixes, RBD feature work surrounding the new object map feature, more CephFS snapshot fixes, and a few important CRUSH fixes.
This is the first development release for the Infernalis cycle, and the first Ceph release to sport a version number from the new numbering scheme. The "9" indicates this is the 9th release cycle--I (for Infernalis) is the 9th letter. The first "0" indicates this is a development release ("1" will mean release candidate and "2" will mean stable release), and the final "0" indicates this is the first such development release.
A few highlights include: