History#

0.2.18 (2023-09-20)#

  • Ignore non-existent columns (GH#290)

  • Add experimental fragments functionality (GH#282, GH#291)

  • Add rechunk by size utility funtion (GH#284)

  • Run CI weekly on Monday @ 2h30 am UTC (GH#288)

  • Update minio server and client versions (GH#287)

  • Retain ROWID coordinates during MS conversion (GH#286)

0.2.17 (2023-08-02)#

  • Change setmaxcachesize to require a read lock, not a write lock (GH#281)

0.2.16 (2023-05-26)#

  • Update calver versioned software such as dask and xarray to 2023 variants (GH#279)

  • Remove unused requirements_dev.txt (GH#275)

  • Support optional CASA columns (GH#270)

0.2.15 (2022-10-19)#

  • Fix poetry install and cache hit detection on CI (GH#266)

  • Support –taql-where for Measurement Sets in dask-ms convert (GH#264)

  • xds_from_zarr should always open zarr groups in read mode (GH#262)

  • Fail on reads if non-existent or invalid store type found (GH#259, GH#260)

0.2.14 (2022-10-04)#

  • Fix for nan chunks/dims breaking writes (GH#255)

  • Fix minio deb download URL (GH#257)

  • Apply black to the code base (GH#252)

  • Manage dask-ms with python poetry (GH#250)

  • Don’t force conversion to tuple during attribute decoding (GH#248)

  • Fail if zarr chunks exceed 2GB (GH#247)

  • Add a --exclude argument to dask-ms convert that allows columns to be excluded during conversion (GH#246).

  • Make --output a required dask-ms convert argument (GH#245).

0.2.12 (2022-09-09)#

  • Consolidate metadata by default during xds_to_zarr and always attempt to read consolidated metadata in xds_from_zarr (GH#243).

  • Fail if ROWID dimensions don’t match associated dataset arrays (GH#239)

  • Pin flake8 to >= 4.0.0 and < 5.0.0 (GH#240)

0.2.11 (2022-07-27)#

  • Improve chunking in xds_to_zarr when rechunk==True. (GH#236)

  • Assign custom fsspec storage options from url match in yaml configuration files. (GH#237)

0.2.10 (2022-07-18)#

  • Minor DaskMSStore changes to improve URL handling. (GH#233)

  • Deprecate Python 3.7 support and test on Python 3.10 (GH#232)

  • Support loading of storage_options in dask-ms convert (GH#231)

  • Reintroduce group columns on output CASA format only (GH#230)

  • Stop converting results returned from DaskMSStore into Path objects (GH#229)

  • Accept storage_options in dataset read/write methods (GH#228)

  • Add __repr__ and __str__ for DaskMSStore (GH#224)

0.2.9 (2022-06-28)#

  • Recreate Grouping Columns when writing to CASA format in dask-ms convert (GH#222)

  • Ignore SOURCE subtable in dask-ms convert (GH#221)

  • Fix performance bug in cached_array implementation. (GH#220)

  • Use DaskMSStore throughout dask-ms convert (GH#218)

  • Upgrade from deprecated visit_Num to visit_Constant (GH#217)

  • Ensure url and table consistency in DaskMSStore (GH#216)

  • Wait for minio to start with greater precision (GH#215)

  • Chunk correctly when reading from parquet. (GH#210)

  • Fix minor bugs in zarr and conversion functionality. (GH#208)

  • Add xds_to_storage_table. (GH#207)

  • Add option to rechunk automatically on writes. (GH#204)

  • Raise more informative error. (GH#203)

  • Improve tiling. (GH#202)

  • Do not create spurious fields in zarr writes. (GH#200)

  • Error out when missing datavars should be written. (GH#197, GH#198)

  • Allow non-standard columns to be tiled. (GH#196)

0.2.8 (2022-04-06)#

  • Fix roundtripping of boolean tensor arrays. (GH#194)

  • Ignore unsupported kwargs in xds_from_storage_* functions. (GH#192)

  • Improve handling of subtables with variably sized rows in daskms-convert. (GH#191)

  • Ensure that xds_from_zarr sorts groups as integers and not strings (GH#188)

  • Ensure Natural Ordering for parquet files (GH#183)

  • Fix xds_from_zarr and xds_from_parquet chunking behaviour (GH#182)

  • Add LazyProxy and LazyProxyMultiton patterns to dask-ms (GH#177)

  • Support cloud native storage formats via fsspec (GH#174)

0.2.7 (2022-01-13)#

  • Fix inclusion of MANIFEST.in files (GH#173)

  • Add –group-columns to dask-ms convert for CASA Formats (GH#169)

  • Add ComplexArray -> numpy conversion (GH#168)

  • Ignore row dimension when fixing column shapes (GH#165)

  • Bump pip from 9.0.1 to 19.2 (GH#164)

  • Fix zarr coordinate writes (GH#162)

  • Deprecate Python 3.6 (GH#161)

  • Add IMAGING_WEIGHT_SPECTRUM to default Measurement Schema (GH#160)

  • Remove default time ordering from xds_from_ms (GH#156)

  • Make zarr writes completely lazy (GH#157)

  • Copy partitioning information when writing (GH#155)

  • Add a dask-ms convert script for converting between CASA, Zarr and Parquet formats (GH#145)

  • Convert code-base to f-strings with flynt (GH#144)

  • Consolidate Dataset Types into daskms.dataset (GH#143)

  • Correct Dataset persistence issues (GH#140)

  • Experimental arrow support (GH#130, GH#132, GH#133, GH#135, GH#136, GH#138, GH#145)

  • Experimental zarr support (GH#129, GH#133, GH#139, GH#142, GH#150, GH#145)

  • Test data fix (GH#128)

  • Fix array inlining for writes (GH#126)

  • Allow Multi-Layer Inlining (GH#125)

  • Support DATA Column Expressions (GH#124, GH#134, GH#146, GH#147, GH#148, GH#151)

0.2.6 (2020-10-20)#

  • Remove table close in ThreadPool for the last time (GH#122)

  • Respect the High Level Graph specification better during inline array creation (GH#123)

  • Support dictionary writes via putvarcol (GH#119)

  • Use getcell instead of getcellslice in sorted orderings (GH#120)

  • Update to pytest-flake8 1.0.6 (GH#117)

  • Test on Python 3.8 (GH#116)

  • Depend on python-casacore 3.3.1 (GH#116)

0.2.5 (2020-05-11)#

  • Remove deadlock in TableProxy weakref.finalize on Python 3.6 (GH#113)

  • Use python-casacore wheels for travis testing, instead of kernsuite packages (GH#115)

0.2.4 (2020-04-24)#

  • Documentation updates (GH#110)

  • Provide better warnings for unusual ROWID graphs during table updates (GH#108)

  • Work around casacore getcolslice caching (GH#107)

  • Update LICENSE year (GH#105)

  • Update license and production status in pypi classifiers (GH#104)

  • Use WHERE rather than HAVING clause in group ordering TAQL (GH#98)

  • Improve the dask task key names for clearer graph visualization (GH#102)

  • Cache and inline row runs in write operations (GH#96)

  • Support getcolslice and putcolslice in TableProxy (GH#91)

  • Use weakref.finalize to cleanup TableProxy and Executor objects (GH#89)

  • Pickle Executor key argument (GH#88)

  • Deprecate Python 3.5 support and test on Python 3.7 (GH#87)

  • Optionally expose TableProxy objects in dataset read/write methods (GH#85)

  • Upgrade to python-casacore 3.2 (GH#84)

  • Re-introduce xarray handling in dataset.as_variable (GH#83)

  • Explicitly require dask Arrays on write datasets (GH#83)

  • Document python-casacore install process (GH#80, GH#81)

0.2.3 (2019-12-09)#

  • Remove __future__ import (GH#79)

  • Update examples (GH#78)

  • Only log aggressively when the log level is DEBUG (GH#76)

  • Optimise dask graphs produced by dask-ms such that each data access node no longer has common ancestors but is instead an independent root node. This improves memory usage in case of the predict. (GH#75)

  • Read-lock TAQL row reference table by default (GH#74)

  • Produce write datasets rather a single concatenated dask array (GH#70, GH#72)

0.2.2 (2019-10-25)#

  • Fix spacing in TAQL WHERE queries (GH#68)

0.2.1 (2019-10-23)#

  • Constrain table object access to Executor. Simplify table locking (GH#66).

  • Fix stress test (GH#65)

  • Remove keywords from variable attributes (GH#64)

0.2.0 (2019-09-30)#

  • Fix and test non-standard sub-table creation (GH#60)

  • Improve sub-table creation logic (GH#59, GH#60)

  • Support table and column keywords (GH#58, GH#62)

  • Support concurrent access of multiple independent tables (GH#57)

  • Fix WEIGHT_SPECTRUM schema dimensions (GH#56)

  • Pin python-casacore to 3.0.0 (GH#54)

  • Drop python 2 support (GH#51)

  • Simplify Table Schemas (GH#50)

  • Add Concepts + Tutorial Documentation (GH#48)

  • Supporting reading and updating column keywords (GH#48)

  • Add OBSERVATION, FEED, POINTING, SOURCE table schemas (GH#48)

  • Remove single row squeezing in the group_cols=”__row__” case (GH#48)

  • Handle multi-dimensional string arrays (GH#48)

  • Add preliminary example_ms (GH#48)

  • Add Concepts + Tutorial Documentation (GH#48)

  • Make xarray an optional dependency (GH#45)

  • Rename xarray-ms to dask-ms (GH#43)

  • Allow chunking by arbitrary dimensions (GH#41)

  • Add a simple Dataset, making xarray an optional dependency. (GH#41, GH#46, GH#47, GH#52)

  • Add support for writing new tables from Datasets (GH#41, GH#53)

  • Add support for appending to tables from Datasets (GH#41)