ContextQMD
Back to Kubo

go-ipfs changelog v0.10

docs/changelogs/v0.10.md

0.41.050.9 KB
Original Source

go-ipfs changelog v0.10

v0.10.0 2021-09-30

We're happy to announce go-ipfs 0.10.0. This release brings some big changes to the IPLD internals of go-ipfs that make working with non-UnixFS DAGs easier than ever. There are also a variety of new commands and configuration options available.

As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our release process for details.

πŸ›  TLDR: BREAKING CHANGES

  • ipfs dag get
    • default output changed to dag-json
    • dag-pb (e.g. unixfs) field names changed - impacts userland code that works with dag-pb objects returned by dag get
    • no longer emits an additional new-line character at the end of the data output
  • ipfs dag put
    • defaults changed to reduce ambiguity and surprises: input is now assumed to be dag-json, and data is serialized to dag-cbor at rest.
    • --format and --input-enc were removed and replaced with --store-codec and --input-codec
    • codec names now match the ones defined in the multicodec table
    • dag-pb (e.g. unixfs) field names changed - impacts userland code that works with dag-pb objects stored via dag put

Keep reading to learn more details.

πŸ”¦ Highlights

🌲 IPLD Levels Up

The handling of data serialization as well as many aspects of DAG traversal and pathing have been migrated from older libraries, including go-merkledag and go-ipld-format to the new go-ipld-prime library and its components. This allows us to use many of the newer tools afforded by go-ipld-prime, stricter and more uniform codec implementations, support for additional (pluggable) codecs, and some minor performance improvements.

This is significant refactor of a core component that touches many parts of IPFS, and does come with some breaking changes:

  • IPLD plugins:
    • The PluginIPLD interface has been changed to utilize go-ipld-prime. There is a demonstration of the change in the bundled git plugin.
  • The semantics of dag put and dag get change:
    • dag get now takes the output-codec option which accepts a multicodec name used to encode the output. By default this is dag-json, which is a strict and deterministic subset of JSON created by the IPLD team. Users may notice differences from the previously plain Go JSON output, particularly where bytes are concerned which are now encoded using a form similar to CIDs: {"/":{"bytes":"unpadded-base64-bytes"}} rather than the previously Go-specific plain padded base64 string. See the dag-json specification for an explanation of these forms.
    • dag get no longer prints an additional new-line character at the end of the encoded block output. This means that the output as presented by dag get are the exact bytes of the requested node. A round-trip of such bytes back in through dag put using the same codec should result in the same CID.
    • dag put uses the input-codec option to specify the multicodec name of the format data is being provided in, and the store-codec option to specify the multicodec name of the format the data should be stored in at rest. These formerly defaulted to json and cbor respectively. They now default to dag-json and dag-cbor respectively but may be changed to any supported codec (bundled or loaded via plugin) by its multicodec name.
    • The json and cbor multicodec names (as used by input-enc and format options) are now no longer aliases for dag-json and dag-cbor respectively. Instead, they now refer to their proper multicodec types. cbor refers to a plain CBOR format, which will not encode CIDs and does not have strict deterministic encoding rules. json is a plain JSON format, which also won't encode CIDs and will encode bytes in the Go-specific padded base64 string format rather than the dag-json method of byte encoding. See https://ipld.io/specs/codecs/ for more information on IPLD codecs.
    • protobuf is no longer used as the codec name for dag-pb
    • The codec name raw is used to mean Bytes in the IPLD Data Model
  • UnixFS refactor. The dag-pb codec, which is used to encode UnixFS data for IPFS, is now represented through the dag API in a form that mirrors the protobuf schema used to define the binary format. This unifies the implementations and specification of dag-pb across the IPLD and IPFS stacks. Previously, additional layers of code for file and directory handling within IPFS between protobuf serialization and UnixFS obscured the protobuf representation. Much of this code has now been replaced and there are fewer layers of transformation. This means that interacting with dag-pb data via the dag API will use different forms:
    • Previously, using dag get on a dag-pb block would present the block serialized as JSON as {"data":"padded-base64-bytes","links":[{"Name":"foo","Size":100,"Cid":{"/":"Qm..."}},...]}.
    • Now, the dag-pb data with dag-json codec for output will be serialized using the data model from the dag-pb specification: {"Data":{"/":{"bytes":"unpadded-base64-bytes"}},"Links":[{"Name":"foo","Tsize":100,"Hash":{"/":"Qm..."}},...]}. Aside from the change in byte formatting, most field names have changed: data β†’ Data, links β†’ Links, Size β†’ Tsize, Cid β†’ Hash. Note that this output can be changed now using the output-codec option to specify an alternative codec.
    • Similarly, using dag put and a store-codec option of dag-pb now requires that the input conform to this dag-pb specified form. Previously, input using {"data":"...","links":[...]} was accepted, now it must be {"Data":"...","Links":[...]}.
    • Previously it was not possible to use paths to navigate to any of these properties of a dag-pb node, the only possible paths were named links, e.g. dag get QmFoo/NamedLink where NamedLink was one of the links whose name was NamedLink. This functionality remains the same, but by prefixing the path with /ipld/ we enter data model pathing semantics and can dag get /ipld/QmFoo/Links/0/Hash to navigate to links or /ipld/QmFoo/Data to simply retrieve the data section of the node, for example.
    • β„Ή See the dag-pb specification for details on the codec and its data model representation.
    • β„Ή See this detailed write-up for further background on these changes.

β“‚ Multibase Command

go-ipfs now provides utility commands for working with multibase:

console
$ echo -n hello | ipfs multibase encode -b base16 > file-mbase16
$ cat file-mbase16
f68656c6c6f

$ ipfs multibase decode file-mbase16
hello

$ cat file-mbase16 | ipfs multibase decode
hello

$ ipfs multibase transcode -b base2 file-mbase16
00110100001100101011011000110110001101111

See ipfs multibase --help for more examples.

πŸ”¨ Bitswap now supports greater configurability

This release adds an Internal section to the configuration file that is designed to help advanced users optimize their setups without needing a custom binary. The Internal section is not guaranteed to be the same from release to release and may not be covered by migrations. If you use the Internal section you should be making sure to check the config documentation between releases for any changes.

🐚 Programmatic shell completions command

ipfs commands completion bash will generate a bash completion script for go-ipfs commands

πŸ“œ Profile collection command

Performance profiles can now be collected using ipfs diag profile. If you need to do some debugging or have an issue to submit the collected profiles are very useful to have around.

🍎 Mac OS notarized binaries

The go-ipfs and related migration binaries (for both Intel and Apple Silicon) are now signed and notarized to make Mac OS installation easier.

πŸ‘¨β€πŸ‘©β€πŸ‘¦ Improved MDNS

There is a completed implementation of the revised libp2p MDNS spec. This should result in better MDNS discovery and better local/offline operation as a result.

πŸš— CAR import statistics

dag import command now supports --stats option which will include the number of imported blocks and their total size in the output.

πŸ•Έ Peering command

This release adds swarm peering command for easy management of the peering subsystem. Peer in the peering subsystem is maintained to be connected at all times, and gets reconnected on disconnect with a back-off.

See ipfs swarm peering --help for more details.

Changelog

Contributors

ContributorCommitsLines Β±Files Changed
Daniel MartΓ­42+8549/-6587170
Eric Myhre55+5883/-6715395
Marten Seemann100+1814/-2028275
Steven Allen80+1573/-1998127
hannahhoward18+1721/-67153
Will2+1114/-121718
Andrew Gillis2+1220/-72014
gammazero3+43/-185610
Masih H. Derkani3+960/-8968
Adin Schmahmann25+1458/-31344
vyzo27+986/-35360
Will Scott6+852/-42416
Rod Vagg19+983/-25566
Petar Maymounkov6+463/-17922
web3-bot10+211/-19524
adlrocha1+330/-7515
RubenKelevra2+128/-2102
Ian Davis3+200/-10917
Cory Schwartz3+231/-337
Keenan Nemetz1+184/-712
Randy Reddig2+187/-538
Takashi Matsuda3+201/-27
guseggert4+161/-209
Lucas Molas5+114/-4727
nisdas4+115/-457
Michael MurΓ©6+107/-3324
Richard Ramos2+113/-93
Marcin Rataj12+88/-2413
Ondrej Prazak2+104/-64
Michal Dobaczewski2+77/-283
Jorropo3+9/-754
Andey Robins1+70/-33
Gus Eggert10+34/-3112
noot1+54/-95
Maxim Merzhanov1+29/-241
Adrian Lanzafame1+30/-132
Bogdan Stirbat1+22/-162
Shad Sterling1+28/-31
Jesse Bouwman5+30/-05
Pavel Karpy1+19/-72
lasiar5+14/-105
Dennis Trautwein1+20/-42
Louis Thibault1+22/-12
whyrusleeping2+21/-12
aarshkshah19923+12/-83
Peter Rabbitson2+20/-02
bt902+17/-22
Dominic Della Valle1+13/-12
Audrius Butkevicius1+12/-11
Brian Strauch1+9/-31
Aarsh Shah2+1/-112
Whyrusleeping1+11/-01
Max1+7/-31
vallder1+3/-51
Michael Burns3+2/-63
Lasse Johnsen1+4/-42
snyh1+5/-21
Hector Sanjuan2+3/-22
市川恭佑 (ebi)1+1/-31
godcong2+2/-12
Mathis Engelbart1+1/-21
folbrich1+1/-11
Med Mouine1+1/-11