docs/changelogs/v0.11.md
This patch release fixes a security issue wherein traversing some malformed DAGs can cause the node to panic.
See also the security advisory: https://github.com/ipfs/go-ipfs/security/advisories/GHSA-mcq2-w56r-5w2w
Note: the v0.11.1 patch release contains the Docker compose fix from v0.12.1 as well
| Contributor | Commits | Lines ± | Files Changed |
|---|---|---|---|
| Rod Vagg | 1 | +34/-19 | 2 |
We're happy to announce go-ipfs 0.11.0. This release comes with improvements to the UnixFS Sharding and PubSub experiments as well as support for Circuit-Relay v2 which sets the network up for decentralized hole punching support.
As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our release process for details.
ipfs add -r <directory> may result in different CIDs due to the different DAG representations.Experimental.ShardingEnabled is removed.Swarm.EnableRelayHop is set to true/api/v0/pubsub changed.
Keep reading to learn more details.
Truly big directories can have so many items, that the root block with all of their names is too big to be exchanged with other peers. This was partially solved by HAMT-sharding, which was introduced a while ago as opt-in. The main downside of the implementation was that it was a global flag that sharded all imported directories (big and small).
This release solves that inconvenience by making UnixFS sharding smarter and applies it only to larger directories (i.e. directories that would be at least ~256KiB). This is now the default behavior in ipfs add and ipfs files commands, where UnixFS sharding works out-of-the-box.
This release adds support for the circuit relay v2 protocol based on the reference implementation from go-libp2p 0.16.
This is the cornerstone for maximizing p2p connections between IPFS peers. Every publicly dialable peer can now act as a limited relay v2, which can be used for hole punching and other decentralized signaling protocols.
go-ipfs can now be configured to act as a RelayClient that uses other peers for autorelay functionality when behind a NAT, or provide a limited RelayService to other peers on the network.
Starting with go-ipfs v0.11 every publicly dialable go-ipfs (based on AutoNAT determination) will start a limited RelayService. RelayClient remains disabled by default for now, as we want the network to update and get enough v2 service providers first.
Note: the limited Circuit Relay v2 provided with this release only allows low-bandwidth protocols (identify, ping, holepunch) over transient connections. If you want to relay things like bitswap sessions, you need to set up a v1 relay by some other means. See details below.
Switching to v2 of the relay spec means removal or deprecation of configuration keys that were specific to v1.
Swarm.EnableAutoRelay was replaced by Swarm.RelayClient.Enabled.Swarm.DisableRelay is deprecated, relay transport can be now disabled globally (both client and service) by setting Swarm.Transports.Network.Relay to falseSwarm.EnableRelayHop no longer starts an unlimited v1 relay. If you have it set to true the node will refuse to start and display an error message.RelayV1.Enabled set to true. Be mindful that v1 relays are unlimited, and one may want to set up some ACL based either on PeerIDs or Subnets.We are working towards enabling hole punching for NAT traversal when port forwarding is not possible.
go-libp2p 0.16 provides an implementation of the DCUtR (decentralized hole punching) protocol. It is hidden behind the Swarm.EnableHolePunching configuration flag.
When enabled, go-ipfs will coordinate with the counterparty using a relayed v2 connection, to upgrade to a direct connection through a NAT/firewall whenever possible.
This feature is disabled by default in this release, but we hope to enable it by default as soon the network updates to go-ipfs v0.11 and gains a healthy set of limited v2 relays.
This release fixed some edge cases that were reported by users of the PubSub experiment, getting it closer to becoming a stable feature of go-ipfs. Some PubSub users will notice that the plaintext limitation is lifted: one can now use line breaks in messages published to non-ascii topic names, or even publish arbitrary bytes to arbitrary topics. It required a change to the wire format used when pubsub commands are executed over the HTTP RPC API at /api/v0/pubsub/*, and also modified the behavior of the ipfs pubsub pub command, which now is publishing only a single pubsub message with data read from a file or stdin.
If you use the HTTP RPC API with the go-ipfs-http-client library, make sure to update to the latest version. The next version of js-ipfs-http-client will use the new wire format as well, so you don't need to do anything.
If you use /api/v0/pubsub/* directly or maintain your own client library, you must adjust your HTTP client code. Byte fields and URL args are now encoded in base64url Multibase. Encode/decode bytes using the ipfs multibase --help commands, or use the multiformats libraries (js-multiformats, go-multibase).
Low level changes:
topic passed as URL arg in requests to /api/v0/pubsub/* must be encoded in URL-safe multibase (base64url)data, from, seqno and topicIDs returned in JSON responses are now encoded in multibasefrom now use the same default text representation from go-libp2p and peerid encoder/decoder from libp2p. This means the same text representation as in as in swarm peers, which makes it possible to compare them without decoding multibase./api/v0/pubsub/pub no longer accepts data to be passed as URL, it has to be sent as multipart/form-data. This removes size limitations based on URL length, and enables regular HTTP clients to publish data to PubSub topics. For example, to publish some-file to topic named test-topic using vanilla curl, one would execute: curl -X POST -v -F "stdin=@some-file" 'http://127.0.0.1:5001/api/v0/pubsub/pub?arg=$(echo -n test-topic | ipfs multibase encode -b base64url)'ipfs pubsub pub on the command line no longer accepts variadic data arguments. Instead, it expects a single file input or stream of bytes from stdin. This ensures arbitrary stream of bytes can be published, removing limitation around messages that include \n or \r\n.Addresses.AppendAnnounce is an array of multiaddrs, similar to Addresses.Announce, except it does not override inferred swarm addresses, but appends custom ones to the list.Pubsub.Enabled enables the pubsub system.Ipns.UsePubsub enables IPFS over pubsub experiment for publishing IPNS records in real time.JOSE is a standard for signing and encrypting JSON objects. DAG-JOSE is an IPLD codec based on JOSE and represented in CBOR. Upon encountering the dag-jose multicodec indicator, implementations can expect that the block contains dag-cbor encoded data which matches the IPLD schema from the DAG-JOSE spec.
This work was contributed by Ceramic and acts as a template for future IPFS improvements driven by the real world needs of the IPFS community.
ipfs files rm.context.CancelFunc instead of func() (#257) (ipfs/go-graphsync#257)sys in mock FileInfo (ipfs/go-ipfs-files#39)IDENTITY CIDs gracefullycar/ cli into cmd/car (#233) (ipld/go-car#233)car get-dag command (#232) (ipld/go-car#232)get block to car cli (#230) (ipld/go-car#230)list and filter commands (#227) (ipld/go-car#227)car split command (#226) (ipld/go-car#226)MultihashIndexSorted the default index codec for CARv2IDENTITY CID in IndexSortedtrueMultihasIndexSortednil as Index reader when reading indexless CARv2OpenReader from file does not panic after closureExtractV1File and address commentsblockstore.AllKeysChan errors via contextReadOptions to be set when getting or generating indexioutil.TempFile to simplify file creation in index examplego-wishtraversal package tests to quicktestcodec package tests to quicktestTypedNode with nil type of kind Mapnil typeLink and []byte in printer (#294) (ipld/go-ipld-prime#294)fluent package tests to quicktestdatamodel package tests to quicktestadl package tests to quicktestnode package tests to quicktest| Contributor | Commits | Lines ± | Files Changed |
|---|---|---|---|
| Will | 13 | +73226/-130481 | 43 |
| Masih H. Derkani | 99 | +10549/-5799 | 489 |
| hannahhoward | 43 | +5515/-3293 | 233 |
| Daniel Martí | 60 | +5312/-2883 | 208 |
| Marten Seemann | 175 | +4839/-3254 | 396 |
| Eric Myhre | 73 | +3924/-3328 | 175 |
| Jessica Schilling | 52 | +2709/-2386 | 75 |
| Rod Vagg | 30 | +2719/-1703 | 79 |
| vyzo | 10 | +3516/-177 | 87 |
| Gus Eggert | 64 | +1677/-1416 | 147 |
| Adin Schmahmann | 23 | +1708/-381 | 95 |
| Lucas Molas | 14 | +1557/-365 | 48 |
| Will Scott | 7 | +1846/-15 | 34 |
| Steven Allen | 32 | +537/-897 | 56 |
| Cory Schwartz | 3 | +614/-109 | 12 |
| rht | 3 | +576/-4 | 7 |
| Simon Zhu | 9 | +352/-51 | 16 |
| Petar Maymounkov | 7 | +173/-167 | 23 |
| RubenKelevra | 1 | +107/-188 | 1 |
| jwh | 2 | +212/-80 | 7 |
| longfeiW | 1 | +4/-249 | 10 |
| guseggert | 5 | +230/-21 | 11 |
| Kevin Neaton | 8 | +137/-80 | 13 |
| Takashi Matsuda | 1 | +199/-0 | 5 |
| Andrey Kostakov | 1 | +107/-49 | 2 |
| Jesse Bouwman | 1 | +151/-0 | 7 |
| web3-bot | 39 | +136/-3 | 52 |
| Marcin Rataj | 16 | +62/-57 | 25 |
| Marco Munizaga | 1 | +118/-0 | 2 |
| Aaron Riekenberg | 4 | +64/-52 | 6 |
| Ian Davis | 4 | +81/-32 | 7 |
| Jorropo | 2 | +79/-19 | 6 |
| Mohsin Zaidi | 1 | +89/-1 | 20 |
| Andey Robins | 1 | +70/-3 | 3 |
| gammazero | 3 | +40/-25 | 4 |
| Steve Loeppky | 2 | +26/-27 | 3 |
| Dimitris Apostolou | 1 | +25/-25 | 15 |
| Sudarshan Reddy | 1 | +9/-40 | 1 |
| Richard Littauer | 2 | +42/-1 | 3 |
| pymq | 1 | +32/-8 | 2 |
| Dirk McCormick | 2 | +23/-1 | 2 |
| Nicholas Bollweg | 1 | +21/-0 | 1 |
| anorth | 1 | +14/-6 | 2 |
| Jack Loughran | 1 | +16/-0 | 2 |
| whyrusleeping | 2 | +11/-2 | 2 |
| bt90 | 1 | +13/-0 | 1 |
| Yi Cao | 1 | +10/-0 | 1 |
| Max | 1 | +7/-3 | 1 |
| Juan Batiz-Benet | 2 | +8/-2 | 2 |
| Keenan Nemetz | 1 | +8/-0 | 1 |
| muXxer | 1 | +3/-3 | 1 |
| galargh | 2 | +3/-3 | 3 |
| Didrik Nordström | 1 | +2/-4 | 1 |
| Ben Lubar | 1 | +3/-3 | 1 |
| arjunraghurama | 1 | +5/-0 | 1 |
| Whyrusleeping | 1 | +3/-2 | 1 |
| TUSF | 1 | +3/-2 | 3 |
| mathew-cf | 1 | +3/-1 | 2 |
| Stephen Whitmore | 1 | +2/-2 | 1 |
| Song Zhu | 1 | +2/-2 | 1 |
| Michael Muré | 1 | +4/-0 | 1 |
| Alex Good | 1 | +4/-0 | 2 |
| aarshkshah1992 | 1 | +2/-1 | 1 |
| susarlanikhilesh | 1 | +1/-1 | 1 |
| falstack | 1 | +1/-1 | 1 |
| Michael Vorburger ⛑️ | 1 | +1/-1 | 1 |
| Ismail Khoffi | 1 | +1/-1 | 1 |
| George Xie | 1 | +1/-1 | 1 |
| Bryan Stenson | 1 | +1/-1 | 1 |
| Lars Gierth | 1 | +1/-0 | 1 |