[ANNOUNCE] urbit-v1.6

72 views
Skip to first unread message

joe bryan

unread,
Oct 5, 2021, 4:49:24 PM10/5/21
to urbit-dev
Arvo 420 K
Vere 1.6

This release includes a native Windows build, compatibility with Apple M1
(via rosetta), a new urbit crypto library (urcrypt) and rewrites of the
crypto jets, support for new "multipass" keyfiles, performance improvements
in IPC deserialization, fixes for a serious bug in noun equality and some
minor memory leaks, and general CI improvements.

- native Window build

Thanks to contributor ~locpyl-tidnyd, we now have a general platform
compatibility layer (and alternative to nix for building dependencies),
allowing us to distribute native Windows binaries built with MinGW.

- Apple M1 compatibility

Also thanks to ~locpyl-tidnyd, our official binaries for Darwin can now
be run on both Intel and M1 (Apple Silicon) Macs. And native M1 binaries
can be built using homebrew.

- urcrypt

Thanks to ~fodwyt-ragful, we have a library encapsulating all our crypto
dependencies, new jets for Keccak, and rewrites of the existing crypto jets.

- multipass

Urbit keyfiles now support multiple private keys, paving the way for fast
bootstrap in a Layer2 world.

- bugs

Deep trees of non-equal nouns were erroneously considered equal in certain
conditions. More egregiously, they were then made to be equal through
memory unification (which exists to increase structural sharing). This bug
was introduced a year ago through refactoring (and insufficient testing thereof)
-- thanks to ~wicdev-wisryt for tracking it down.

Memory-leak checking in CI was much weaker than expected, as it only ran on
the persistent results of virtual computations. Thanks to ~fodwyt-ragful for
finding and fixing both the leaks that had sneaked through and the testing
regime to prevent them going forward.

Edward Amsden (2):
      vere: use memset to wipe hashmaps
      Merge pull request #5218 from urbit/ea/memset-to-wipe-dict32

Joe Bryan (73):
      vere: deprioritize %behn
      Merge branch 'master' into release/next-vere
      Merge pull request #4882 from l2dy/safe-malloc
      Merge remote-tracking branch 'locpyl-tidnyd/mingw-port' into jb/rc
      Merge remote-tracking branch 'upstream/m/multipass' into jb/rc
      jets: updates +rip and +rep stub jets to defer to nock
      build: updates libaes_siv, builds from upstream
      build: updates secp256k1 source
      Merge remote-tracking branch 'frodwith/urcrypt' into jb/urcrypt-prep
      Squashed 'pkg/urcrypt/argon2/' changes from 4da94a611e..a4c1e3f713
      Merge commit '0b3f9a629f864ad1e68c5f5a73d11c93ae42d644' into jb/urcrypt-merge
      build: gitignore library binaries
      build: gitignore urcrypt build results
      urcrypt: updates shared-library build test, accounts for mangling
      urcrypt: updates libaes_siv link test, supplying -lcrypto
      build: force static urcrypt on darwin
      build: hack urbit configure for static linking
      build: implements static builds in urbit ./configure, prunes deps
      build: normalize our use of pkg-config, rename urcrypt config
      build: remove obsolete (urcrypt-vendored) dependencies
      build: fixes urbit+urcrypt mingw build
      build: update mingw CI to use "ares" cachix cache
      pmnsh: adds support for pkg-config
      mingw: sets libh2o install dir (for accurate pkg-config info)
      build: restore division between pkg-config and regular deps
      urcrypt: restores pkg-config api versioning scheme
      build: adds urcrypt to top-level shell.nix
      u3: fixes u3kc_bex() refcount, updates style
      u3: fixes sample deconstruction error handling in scrypt jets
      u3: renames secp init/stop functions, fixes comment
      test: removes obsolete scrypt test comment
      u3: refactors address layout macros, combines darwin/aarch64
      Merge branch 'jb/urcrypt' into jb/next-gen-term
      build: updates pmnsh to *prepend* to PKG_CONFIG_PATH
      build: fixes mingw static builds, tweaks urbit configure --enable-static
      Merge branch 'jb/urcrypt' into jb/next-gen-term
      Merge remote-tracking branch 'upstream/m/multipass' into jb/rc
      Merge branch 'master' into release/next-vere
      Merge branch 'jb/rc' into release/next-vere
      pill: all
      Merge pull request #5170 from urbit/jb/urcrypt-prep
      Merge pull request #5171 from urbit/jb/urcrypt-merge
      Merge branch 'release/next-vere' into jb/next-gen-term
      pill: all
      Merge pull request #5174 from urbit/jb/darwin-loombase
      vere: suppress null writes in term.c
      vere: refactors _term_it_send_csi()
      vere: renames terminal streams for clarity, removes unused uv_tcp_t
      build: update pmnsh to support cachix text/x-nix-narinfo responses
      build: patch windows uv_pipe impl to allow null writes
      u3: replaces spurious bloq-size bail:exit's in +rap and +rep jets
      vere: adds backwards-compatibility to blit handling under -t
      vere: adds backwards-compatibility TODO comments for terminal inputs
      serf: fix incorrect conditional (integer precision) in event tracing
      u3: restore hashtable root struct layout for compatibility
      tests: remove unnecessary runtime config from hashtable_tests
      Merge branch 'release/next-vere' into jb/leaky
      Merge pull request #5230 from urbit/jb/leaky
      Revert "Merge pull request #4463 from urbit/m/next-gen-term"
      vere: suppress null writes in term.c
      vere: renames terminal streams for clarity, removes unused uv_tcp_t
      build: update pmnsh to support cachix text/x-nix-narinfo responses
      build: patch windows uv_pipe impl to allow null writes
      u3: fixes printf in c3_assert()
      vere: fixes crash (null deref) in comet booting
      vere: bumps versions
      Merge branches 'jb/rnv-restage' and 'jb/rnv-fixes', remote-tracking branches 'frodwith/keccak' and 'botter-nidnul/release/next-vere' into jb/rnv-merge
      serf: scry for full space profile in static grab
      serf: disable hashboard in grab
      remove (old, wrongfully committed) test binary
      serf: disable hashboard in meld
      serf: updates grab to fallback to old output on error
      Merge pull request #5295 from urbit/jb/serf-whey

Jōshin (2):
      vere: have king chdir to pier directory on boot
      Revert "Merge pull request #5107 from urbit/jo/chdir/next-vere"

Logan Allen (1):
      h2o: update to 2.2.6

Paul Driver (87):
      urcrypt pkg with one function, building
      ed_scalarmult jet only uses urcrypt
      one working function in so
      ed_sign moved to urcrypt
      move ed_point_add to urcrypt
      ed_scalarmult_base -> urcrypt
      urcrypt_ed_add_scalarmult_scalarmult_base
      urcrypt_ed_add_double_scalarmult
      urcrypt_ed_puck
      urcrypt_ed_shar
      urcrypt_ed_veri
      begin move to u3r_unpack() (new function) for consistency in crypto jets
      more unpacks
      rest of the unpacks
      aes_ecb -> urcrypt
      const qualifiers, inplace reverse, aes_cbca
      add u3r_unpack_alloc()
      making the bodies of the cbc functions call helpers to be more boring
      more helper functions for shorter, more boring code in aes ecb, cbc
      cbc:aes -> urcrypt
      argon2
      rename u3r_unpack->u3r_bytes_fit, u3r_unpack_alloc->u3r_bytes_all
      move argon unpacking helpers to u3r
      blake2b
      cleaning up a bit, rethinking things
      overwrite inputs instead of copying in urcrypt
      cleaning up _cbc_pad a bit
      integrate jets with urcrypt changes
      move some includes into urcrypt.c
      ripemd-160->urcrypt
      sha1->urcrypt
      shax->urcrypt
      shay->urcrypt
      shas->urcrypt
      shal->urcrypt
      remove unused old aes c file
      use urcrypt instead of openssl for bash calculation
      name the sha functions consistently
      aes-siva-en->urcrypt
      siva_de
      aes-siv jets all using urcrypt
      add punt logging for aes-siv
      normalize punt logging with u3l_punt
      secp_make->urcrypt
      WIP: returning from secp rectification on master and preparing for rebase and merge.
      stops trying to hide pointer shapes, cleans up secp api, updates secp dependency
      compiles and starts
      fixes were made to the secp code and PR'd against master - updating in preparation to merge those fixes
      merge secp changes
      updating solid pill after merge
      obliterate ge-additions package, move to urcrypt
      Squashed 'pkg/urcrypt/ed25519/' content from commit 7fa6712ef
      Merge commit '9ccdead8d8cdd30d2229bd11ed8d4a40861b4c96' as 'pkg/urcrypt/ed25519'
      mid-autoconfiscation of urcrypt
      Merge commit 'aa02d49a228c62fe13299a3e2bd38b4f7316f927' as 'pkg/urcrypt/argon2'
      Squashed 'pkg/urcrypt/argon2/' content from commit 4da94a611
      more autoconfiscation
      use convenience libraries for independent cflags etc
      more flags fixing, remove some obsolete deps
      whitespace and symbol cleanup
      use srcdir in -I to support out of tree builds
      add autogen.sh
      remove some old files
      require shared ssl when building a shared urcrypt
      cosmetic configure things
      pkg-config support for urcrypt, update urbit build
      finish porting secp jets to urcrypt
      Squashed 'pkg/urcrypt/scrypt/' content from commit a402f4116
      Merge commit 'a06fc1416581772e8eddfa20001d131dea5bed64' as 'pkg/urcrypt/scrypt'
      start scrypt porting
      scr-pbk->urcrypt
      move the rest of the scrypt jets to urcrypt, enable them, and correct the hoon test to match the source rfc.
      remove scrypt from urbit build (in urcrypt now)
      add a versioning scheme to urcrypt
      update urbit's configure to use a liburcrypt version
      clean generated pkg-config file
      check for recovery header presence in configure, put -O3 in flags, move pc to distcleanfiles
      libaes_siv now using tip of dfoxfranke master
      fix typo in urcrypt.h
      move libaes_siv to deps
      fix leak in hashtable
      urcrypt,jets: initial commit of keccak material
      Squashed 'pkg/urcrypt/keccak-tiny/' content from commit 9b438dc0f7
      Merge commit '5513eaf62ae0e19ced43238537bdc81c97eda11d' as 'pkg/urcrypt/keccak-tiny'
      urcrypt: fixing some typos - now builds
      working keccak jets
      jets: ice keccak

Philip Monk (7):
      ames: add ames filter for rescue operations
      vere: run memory checker in grandchildren roads
      jets: fix leak in dif:in,by
      vere: fix leak in _n_comp
      vere: fix leak in hmac
      vere: make hashtable_tests complain about leak
      u3: fix big noun equality

Zero King (1):
      jets: remove check after use

fang (12):
      kh: improve code style
      jael, dawn: support multi-keyfile during boot
      aqua: update for latest dawn-event format
      tests: update dawn tests for recent changes
      dawn: correctly propagate multikeyfile errors
      kh: expose to/fromNoun template logic separately
      kh: support multikeyfiles
      dawn: better multikey format
      tests, aqua: update  for recent changes
      Merge branch 'master' into m/multipass
      kh: improve naming and code style
      kh: don't error during key validation
      Merge pull request #4463 from urbit/m/next-gen-term

ryjm (2):
      king: fix noun derivation
      king: add crud event to jael

~botter-nidnul (1):
      urcrypt: argon2 only optimize for x86_64 on x86_64

~locpyl-tidnyd (76):
      jets: rename jets/c/con.c due to clash with legacy CP/M device names
      vere: request binary mode in fopen/open calls
      vere: cleanup and reorder #includes
      u3: fix typo: show suggested loom base if fixed address is unavailable
      vere: use embedded CA store directly rather than save it to a tempfile
      u3: fix patch cleanup
      u3: centralize use of u3_Signal
      u3: rename function to avoid name clash on MingW
      vere: replace dprintf (missing on MingW) with snprintf+write
      libent: use BCryptGenRandom on Windows
      u3: add U3_OS_PROF macro to gate profiling-related code
      vere: comment out unused code that doesn't compile on MingW
      vere: add -fcommon to be compatible with gcc 10
      vere: lmdb: wrap mdb_strerror in mdb_logerror
      vere: #include termios.h from vere.h where it is actually used
      vere: basic MingW compatibility changes
      vere: select dumb terminal with #define rather than Makefile conditional
      vere: export term.c:_write() as u3_write_fd(), use in dprintf() replacement
      compat: fix .patch file modes
      compat: move build-mingw script into configure
      compat: support build from a tarball that does not contain binary pills
      vere: fix strict builds
      meta: add mingw build action
      vere: mingw: build static executables
      compat: mingw: don't export secp256k1 symbols from exe
      vere: revert d554c18dc35253e0158353932944f8563c34b788, set mingw binary mode using _fmode
      compat: make Makefile more flexible
      vere: terminal compatibility
      vere: terminal compatibility
      vere: allow enable/disable optimizations from ./configure
      vere: more file portability
      vere: fix no-tty mode startup
      vere: pipe serf stderr back to lord
      vere: mingw: terminal compatibility
      compat: mingw: implement utimes
      vere: mingw: basic daemon mode support
      vere: mingw: support invocation w/o .exe suffix
      Merge tag 'urbit-v1.3' into mingw-port
      vere: portability fixes after urbit-v1.3 merge
      vere: merge urbit-v1.4
      vere: merge urbit-v1.5 into mingw-port
      vere: move daemon mode code into compat/*/daemon.c
      vere: boot notification in mingw daemon mode
      vere: mingw: add smoke test to build action
      compat: mingw: optionally cache dependency artifacts in cachix
      compat: mingw: remap dependency directories
      compat: mingw: simplify poor-mans-nix-shell.sh
      compat: mingw: fix cmdmake splitting
      compat: mingw: make smoke test use the same parameters as urbit-tests
      compat: mingw: comment kill(2) stub implementation
      vere: mingw: remove signal(SIGPIPE) calls: Win32 has no SIGPIPE
      u3: remove unused #include
      compat: always add compat dirs to -I
      build-action: temporarily use PR-specific cache
      compat: mingw: catch Ctrl-C in main process
      u3: rsignal refactoring
      compat: mingw: fix signal numbers
      compat: mingw: single-thread rsignal implementation
      vere: mingw: pass Ctrl-C to serf
      compat: mingw: make sure .patch files check out without CRLF conversion
      compat: mingw: default to release build
      Merge master into mingw-port to make typescript-check.yml happy
      u3: Makefile: include compat/*/*.mk
      mingw: add rsignal_raise to rsignal.h; raise on same thread
      vere: get rid of curl typecheck warnings for CURLOPT_WRITEFUNCTION
      vere: mingw: bump curl version and get rid of CURLOPT_SSL_CTX_FUNCTION typecheck warning
      vere: mingw: get rid of libsigsegv
      vere: mingw: fix seh_handler_decorator.mk
      vere: ignore more intermediate build files
      compat: fix missing newlines at end of file
      vere: update argon2u dependency to urbit/argon2@a4c1e3f7
      compat: mingw: add seh_handler_decorator comments, move SEH handler to compat/mingw/seh_handler.c
      compat: mingw: put downloaded dependencies under $NIX_STORE (default to pkg/build so that git ignores them)
      compat: mingw: fix dependency patch path
      compat: add m1brew
      compat: minor refactoring

~master-morzod
Reply all
Reply to author
Forward
0 new messages