Chainer v6.0.0rc1 をリリースしました

27 views
Skip to first unread message

iman...@preferred.jp

unread,
Apr 4, 2019, 4:54:11 AM4/4/19
to Chainer Japanese User Group

Chainer v6.0.0rc1 をリリースしました！リリースノートは以下の通りです．


Announcements

  • After this release, the master branch is switched to the development of v7 series. v6.0.0 will continue developing at the v6 branch.
  • (#6629) You can now access the product backlog (the task list that ChainerX core team is willing to work on) as a spreadsheet here. Note that the sheet is actively edited by ChainerX core dev team. The items are NOT promises; we may drop any features from the list any time, but you can use it to know in which direction the development is going on in the near future.

Highlights

  • Mixed precision training support is improved.
    • In particular, mixed precision mode (a.k.a. mixed16 dtype) is added. You can set an environment variable CHAINER_DTYPE=mixed16 to make Chainer choose appropriate dtypes for mixed precision training (in most places it is float16, but it automatically chooses float32 when it’s better for precision and performance reasons).
    • Loss scaling for avoiding underflow in backprop with float16 now supports dynamic mode. In this mode, the scaling factor is adjusted during training so that backprop does not overflow. You can use it with (optimizer).loss_scaling(). See the documentation for details.

Changes without compatibility

  • Deprecate old NCCL versions and related communicators (#6506)
    • Support of NCCL<2.3 is deprecated. We encourage users to use NCCL 2.3 or later ones.

New Features

  • Human readable representation of link and chain (#4853, thanks @wkentaro!)
  • Add variable.item() (#5797, thanks @crcrpar!)
  • Refactor Link.to_device family (#5986)
  • Add decorrelated batch normalization (#6150, thanks @crcrpar!)
  • Add option unit to CupyMemoryProfileHook.print_report() (#6256, thanks @hitsgub!)
  • Add distributions.Independent (#6324, thanks @ganow!)
  • Dynamic loss scaling (#6337, thanks @anaruse!)
  • Add ChainerX FloorDivide (#6350)
  • Customizable forward output check in testing.FunctionTestCase (#6444)
  • Adding fp16 support to the ChainerMN communicators (#6448)
  • mixed16 mode and its support in L.BatchNormalization (#6456)
  • Add shape and dtype check before allrecuce (#6461)
  • Add F.relu6 as an alias to F.clipped_relu (#6463, thanks @aksub99!)
  • Implementation of sigmoid for ChainerX (#6472, thanks @dido1998!)
  • Add minimum to chainerx (#6477, thanks @aksub99!)
  • Add square to chainerx (#6486, thanks @aksub99!)
  • Add chainerx.testing.integral_dtypes (#6526)
  • Support for chainer.mixed16 data type in PureNcclCommunicator (#6548)
  • Add LinkTestCase to simplify link tests (#6559)
  • Add Sin and Cos to chainerx (#6601, thanks @kshitij12345!)
  • Support for fp16 and mixed16 in MultiNodeBatchNormalization of ChainerMN (#6619)
  • Add tan, arcsin, arccos, arctan to ChainerX (#6703, thanks @IvanYashchuk!)

Enhancements

  • Improve F.resize_images speed (#5753, thanks @grafi-tt!)
  • Improve F.group_normalization via cuDNN call (#5924, thanks @grafi-tt!)
  • Fix backward of F.average_pooling_nd with pad_value of None (#6332, thanks @crcrpar!)
  • Support for fp16 in naive comm (#6333)
  • Change backward of F.log_ndtr to avoid NaN (#6340)
  • Stop retaining y.grad on y.backward(retain_grad=False) (#6348)
  • Set requires_grad explicitly in gradient_check and function test (#6364)
  • Fix error messages in get_fans (#6365)
  • ChainerX dtype promotion: mathematical functions (#6379)
  • Mixed dtype: concatenate (#6381)
  • ResultType to take kind into account (#6419)
  • Improve FunctionTestCase error message (#6426)
  • Mixed dtype: arithmetics (#6432)
  • Change intermediate dtype of Adam for float16 parameters to float32 (#6442)
  • Mixed dtype: dot (#6443)
  • Avoid using pytest attributes during import (#6453)
  • Dot product for higher dimensions chainerX (#6476, thanks @dido1998!)
  • Remove dtype from chainerx.Scalar (#6481)
  • Mixed dtype: BatchNorm and FixedBatchNorm (#6484)
  • Support chainerx::Take indices other dtype than int64 (#6485)
  • Keep backward compatibility on cupy.cudnn.batch_normalization_forward_training (#6497)
  • Deprecate old NCCL versions and related communicators (#6506)
  • Mixed dtype chainerx::conv and chainerx::conv_transpose (#6510)
  • Support non-float cast in F.cast (#6518)
  • Remove restriction of x.dtype == b.dtype in F.convolution_nd and F.deconvolution_nd (#6524)
  • Avoid exposing chainerx.Scalar to Python (#6535)
  • Fix parameterize_pytest to allow parameterizing with tuples (#6554)
  • Change device spec (#6563)
  • Mixed dtype support in chainerx.linear (#6569)
  • Check lengths of args of chainer.grad (#6580)
  • Mixed dtype: comparison (#6590)
  • Fix linspace (#6605, thanks @kshitij12345!)
  • Add PerformanceWarning (#6617)
  • Implemented ChainerX version of Clipped ReLU forward (#6627, thanks @Harshan01!)
  • Allow comma separated keys in testing.product (#6635)
  • BatchNormalization to only allocate dummy mean and var in cuDNN path (#6656)
  • Generate shorter class names for parameterized tests (#6660)
  • ChainerX dynamic op registry (#6675)
  • Remove unnecessary broadcasts from F.layer_normalization (#6680, thanks @hitsgub!)
  • Remove unnecessary broadcasts from F.l2_normalization (#6681, thanks @hitsgub!)
  • Support cupy-cuda101 package (#6700)
  • Properly handle FP16 in D.Normal (#6709)
  • Mixed-dtype: minimum and maximum (#6713)
  • Op registration: indexing (#6718)
  • Op registration: logic (#6727)
  • Op registration: trigonometric (#6729)

Bug Fixes

  • Forbid calling empty Sequential (#6304)
  • Fix fp16 issue in batch normalization (#6323, thanks @anaruse!)
  • Fix F.softmax_cross_entropy float16 under/overflow (#6366)
  • Fix lazy init of BatchNormalization link (#6369)
  • Fix str.join TypeError in FunctionTestCase helper (#6370)
  • Fix chainer.links.NStepRNN and its variants (#6415, thanks @crcrpar!)
  • Fix an off-by-one in slicing of chainerx::Array (#6540)
  • Fix more corner cases in chainerx::Slice (#6557)
  • Fix dimension check of chainerx::Linear (#6593, thanks @crcrpar!)
  • Fix ChainerX optimzer fallback for non-default devices (#6699)
  • Fix DeviceResident.to_gpu fallback argument (#6712)

Code Fixes

  • Fix F632 (use == / != to compare str) (#6346)
  • Avoid # NOQA in docstrings (cont.) (#6356)
  • Fix comment style of op_utils.py (#6421)
  • Refactor chainerx::Linear (#6425)
  • Fix ResultTypeResolver multiple definitions (#6439)
  • Assert that input to array props formatter is a list or tuple (#6440)
  • Fix style of .clang-tidy (#6445)
  • Remove unnecessary AsContiguous in CudaConv::ConvGradWeight (#6520)
  • Remove commented out code from _BNMode (#6582)
  • Change the deprecated collections (#6645)
  • Remove obsolete assertions (#6648)
  • Allow ArrayBody::GetArrayNode to return null (#6658)
  • Make BackwardBuilder::Target less stateful (#6659)
  • Clean up test code (#6688)

Documentation

  • Write guides to implement new-style functions (#4986)
  • Fix typo (#6384, thanks @aksub99!)
  • Fix Sphinx markups in RNNs docs (#6412, thanks @crcrpar!)
  • Fix docment in TimerHook (#6433, thanks @hitsgub!)
  • Refactor documentation of F.prelu (#6455, thanks @fiarabbit!)
  • Fixes typo in docstring for classification_summary (#6515, thanks @yewang!)
  • Write TODOs to address Dot backward cast (#6537)
  • Override forward in LinkHook documentation (#6546, thanks @crcrpar!)
  • Remove duplicated entry in reference (#6571)
  • Fix F.rrelu documentation (#6581, thanks @fiarabbit!)
  • Add gradient_check.check_double_backward in reference (#6584)
  • Fix :meth: link (#6603, thanks @23pointsNorth!)
  • Update broken link in chainerx.md (#6610, thanks @kshitij12345!)
  • Improve docs and exception message in F.erfcx, F.erfcinv and F.erfinv (#6618)
  • Include a link to ChainerX product backlog (#6630)
  • Fix missing module declaration (#6662)
  • Fix chainer.backend.get_array_module documentation (#6663)
  • Fix typo: 'Notatition' -> 'Notation' (#6673, thanks @nai62!)
  • Fix test failures in FunctionNode implementation guide (#6734)

Installation

  • Environment variable to set ChainerX Python binding build type (#6647)
  • Check CMAKE_BUILD_TYPE (#6664)

Examples

  • Use args.out in train_cifar_custom_loop.py (#6378, thanks @crcrpar!)
  • Fix to use right device for DALI iterator in imagenet example (#6397)
  • Properly pass device ID to DALI pipelines in imagenet example (#6429)
  • Use __future__.division in imagenet example with Python2 (#6462)
  • Fix broken imagenet example (#6489)
  • Fix wavenet example to support the default dtype (#6536)
  • Use float division instead of __future__.division for Python2 (#6562)
  • Fix DCGAN example to work with default dtype (#6585)
  • Use F.matmul instead of F.batch_matmul in memnn example (#6611)
  • Remove unnecessary unchain_backward() in pix2pix example (#6634, thanks @hayato-maki!)
  • Fix file mode of mushrooms.csv (#6693)
  • Replace deprecated URLopener in download.py (#6694)

Tests

  • Test all codes in guides/functions.rst (#6194)
  • Test various spatial_scale for roi_average_pooling_2d (#6238, thanks @knorth55!)
  • Test simplifications
  • Fix examples/.gitignore (#6391, thanks @crcrpar!)
  • Suppress warning in caffe test (#6402)
  • Add ChainerX test to FunctionTestCases (#6416)
  • Remove SPHINXOPTS env from Makefile (#6417)
  • Rewrite ChainerX connection tests (#6424)
  • Fix regex in test_print_report (#6430)
  • Fix duplicated test (#6434)
  • Add strides check in NumpyOpTest (#6437)
  • Rewrite ChainerX indexing tests (#6438)
  • Add float16 and float 64 to F.group_normalization test (#6468, thanks @crcrpar!)
  • Rewrite ChainerX linalg tests (#6469)
  • Fix F.pad test for Python2 (#6478)
  • Fix input of F.vstack to a list of ndarrays (#6494, thanks @crcrpar!)
  • Change pytest version requirement (#6502)
  • Force camel case class name for OpTest (#6507)
  • Test result dtype permutation (#6511)
  • Fix test class name (#6532)
  • Rewrite ChainerX batch_norm test (#6542)
  • Rewrite ChainerX sorting tests (#6550)
  • Rewrite ChainerX logic tests (#6551)
  • Rewrite ChainerX activation tests (#6553)
  • Rewrite ChainerX manipulation tests (#6556)
  • Rewrite ChainerX fixed_batch_norm test (#6558)
  • Rewrite ChainerX pooling tests (#6560)
  • Rewrite ChainerX arithmetics tests (#6566)
  • Rewrite ChainerX math tests (#6568)
  • Fix tolerance in chainerx.divide test (#6573)
  • Improve arithmetics tests (#6577)
  • Adjust tolerances of F.einsum tests (#6588)
  • Check grads of inputs to test backward of collective communication (#6589)
  • Avoid mutating FunctionTestBase class attributes (#6599)
  • Avoid mutating LinkTestCase and LinkInitializersTestCase class attributes (#6600)
  • Make op_test decorator remove the previous class (#6602)
  • Use compute_60 instead of compute_50 to run test on P100 (#6633)
  • Destroy NCCL communicator after every use (#6636)
  • Run ChainerX python tests in debug build (#6649)
  • Suppress numpy warnings in math tests (#6651)
  • Fix testing condition of BatchNormalizationMultiGpuTest (#6652)
  • Remove C++ routines tests (#6667)
  • Minimize the Travis CI matrix (#6677)
  • Fix conflicts between 6432 and 6486 (#6679)
  • Stop clang-tidy test in Travis CI (#6682)
  • Fix tolerance in TestConvTranspose (#6691)
  • Rewrite the rest of math tests (#6695)
  • Fix test failure in cuDNN v7.5 (#6710)
  • Fix F.convolution_nd test for flake8 (#6711)
  • Relax tolerances in convolution_nd function test (#6728)
Reply all
Reply to author
Forward
0 new messages