Testing, benchmarking, performance improvements

.github/workflows/ci.yml

@@ -1,19 +1,55 @@
+name: Continuous integration
+
 on:
   push:
-    branches: [ main ]
-  pull_request:
+    branches: [main, f/improvements]
+  workflow_dispatch:
 
-name: Continuous integration
+permissions:
+  contents: read
 
 jobs:
   ci:
-    runs-on: ${{ matrix.runner }}
-    strategy:
-      matrix:
-        runner: [a40, orin]
+    name: A40 CUDA and Python tests
+    runs-on: [self-hosted, linux, x64, a40]
+    timeout-minutes: 40
+
+    env:
+      JOBS: "4"
+      SANITIZER_REPEATS: "3"
+      COMPUTE_SANITIZER: /usr/local/cuda/bin/compute-sanitizer
+
     steps:
-      - name: checkout code
-        uses: actions/checkout@v4
-      - name: run test script
-        run: bash ./ci/script.sh
+      - name: Check out code
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+
+      - name: Check CUDA
+        run: |
+          nvidia-smi
+          /usr/local/cuda/bin/nvcc --version
+          /usr/local/cuda/bin/compute-sanitizer --version
+
+      - name: Prepare Python environment
+        run: |
+          python3 -m venv build/test-venv
+          build/test-venv/bin/python -m pip install --upgrade pip
+          build/test-venv/bin/python -m pip install ./python pytest
+
+      - name: Build, run unit tests, and check CUDA memory
+        run: |
+          bash scripts/check_memory.sh -DSM_ARCH=86 -DCPPVERSION=20
+
+      - name: Run Python tests with pytest
+        run: bash scripts/run_python_tests.sh
 
+      - name: Save diagnostic logs
+        if: always()
+        uses: actions/upload-artifact@v7
+        with:
+          name: a40-test-logs
+          path: |
+            build/test/Testing/Temporary/
+            build/sanitizer/
+          if-no-files-found: ignore

.gitignore

@@ -269,4 +269,23 @@ __pycache__/
 */build/
 
 
-cmake-*
+cmake-*
+
+# CMake/CTest artefacts (including accidental in-source builds)
+CMakeCache.txt
+CMakeFiles/
+CTestTestfile.cmake
+Testing/
+Makefile
+cmake_install.cmake
+install_manifest.txt
+**/_deps/
+
+# Build products from this project
+/gputils_main
+/lib/
+/test/gputils_test*
+
+# Temporary files produced by the C++ file-I/O tests
+/myTest.bt
+/myTest.dtensor

CHANGELOG.md

@@ -5,6 +5,68 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+<!-- ---------------------
+      v2.0.0
+     --------------------- -->
+## v2.0.0 - 04-11-2026
+
+### Added
+
+- `scripts/run_python_tests.sh` runs the Python pytest suite with configurable interpreter/build paths and optional C++ exchange tests.
+- File-I/O tests cover shapes, precision, binary layout, non-contiguous inputs, malformed files, filesystem errors, and actual C++/Python exchanges using temporary fixtures.
+- `LeastSquaresBatchedWorkspace<T>` preallocates reusable status storage and one-entry matrix-pointer arrays only for batches of one.
+- `Svd::rankRelative(rtol, atol = 0)` adds GPU rank calculation using per-matrix thresholds `max(atol, rtol * sigmaMax)` and validated tolerances.
+- Build/test and CUDA memory-check scripts run C++/Python tests, repeated leak checks, and save sanitizer logs.
+- Size-safe `Session::recordAllocation(size_t)`, `recordDeallocation(size_t)`, and counted `cudaRelease` methods.
+- `DTensor::getRowsInto(destination, rowsFrom, rowsTo, matIdx)` validates and copies rows into preallocated storage on the source stream.
+- `DTensor::leastSquaresBatched(b, infoArray)` supports reusable device status storage without per-call allocation.
+- `Session::numStreams()` exposes the initialized stream count.
+- Configurable GPU benchmarks compare batched GEMM and row-extraction methods, with correctness checks, host/GPU timings, and CSV results.
+- Regression tests cover validation, files, GEMM coefficients, stream ordering, ownership, rank tolerances, and reusable buffers.
+
+### Changed
+
+- **Breaking:** replace `incrementAllocatedBytes(int)` with separate size-safe allocation/deallocation methods.
+- **Breaking:** copy assignment deep-copies; move assignment transfers storage and empties the source. Use slices for aliases and `deviceCopyTo` for preallocated copies.
+- Reshape retains and reuses owned matrix-pointer capacity where possible.
+- Column-major uploads use the supplied host vector directly; only row-major uploads allocate a conversion buffer.
+- `setStreamIdx` returns a reference for lvalues and moves rvalues, avoiding implicit allocation and deep copying.
+- `getRows` uses one asynchronous 2D copy and inherits the source stream; `getRowsInto` preserves the destination's stream index.
+- Tensor constructors validate dimensions, allocation overflow, vector length, and storage mode before allocating.
+- Validate reshape sizes, slices, random bounds, row extraction, debug access, empty factoriser inputs, and QR output shapes.
+- Reject zero streams and reconfiguration after initialization; validate stream indices against the initialized count.
+- Document numerical preconditions, buffer lifetimes, stream dependencies, reusable workspaces, and binary-format assumptions.
+- Ignore generated build files, dependencies, binaries, and temporary test artifacts, including in-source builds.
+
+### Fixed
+
+- Nullspace projection uses constructor-allocated scratch and stream-ordered copy-back to avoid GEMM aliasing; tests cover dense batches, repeated calls, streams, and allocation reuse.
+- Givens annihilation uses the identity rotation for an already-zero pair, avoiding NaNs and preserving both rows; covered for float and double.
+- Python file I/O rejects incomplete headers, payload-size mismatches, zero dimensions, empty/scalar arrays, and object/zero-size dtypes before loading or overwriting data.
+- Python binary loading preserves column-major element order for single matrices, with a rectangular round-trip regression test.
+- Batched Cholesky and least-squares support single matrices using constructor-allocated pointer storage without allocations during reusable solves.
+- Size QR workspace for the largest requirement across factorisation, least-squares, and Q extraction; add large square/tall regressions for both precisions.
+- Share stream configuration across translation units using an inline static `Session` member, with a two-file regression test.
+- Prevent SVD constructor exception leaks using a `unique_ptr` owner and raw observer; verify cleanup with allocation-failure tests.
+- Tensor constructors release GPU storage on subsequent host-allocation failures; fault-injection tests verify cleanup for float, double, and int.
+- Session shutdown completes pending work, releases library handles, and destroys owned CUDA streams.
+- Single and batched Cholesky queries, factorisations, solves, and status tensors use the matrix-selected stream.
+- Host transfers, scalar reads, initialization, and pointer uploads complete on their selected stream; transpose results inherit the source stream.
+- Device copies wait for prior destination work and complete on the source stream before returning.
+- QR debugging-helper uploads complete on the selected solver stream before returning, fixing intermittent stream-ordering failures.
+- Track owned bytes and pointer capacity independently of shape, count successful allocations/frees, and preserve accounting across reshape, views, and moves.
+- Reset `Svd::rank()` counters on each call; SVD factorisation, rank kernels, and results use the input stream.
+- Assignment releases previous storage and safely handles metadata, self-assignment, and empty tensors; deep copies complete on the source stream.
+- Enable testing at the project root so `ctest --test-dir build` discovers the test suite.
+- Make the Python-format binary-reader test independent of generated Python files and the working directory.
+- Reject malformed text and incomplete or incorrectly sized binary files; report write failures and handle short filenames and empty data safely.
+- Avoid null-pointer arithmetic when slicing a single-matrix tensor along the matrix axis.
+- QR operations use the matrix-selected cuSolver/cuBLAS stream; `getQR` synchronizes before reading R on the host.
+- Givens coefficient kernels use the selected cuBLAS stream, check launch errors, and restore the previous pointer mode.
+- Make test-stream setup independent of execution order and add nonblocking-stream regressions for QR, Givens, and row extraction.
+- Correct the row-slicing test to select a column before slicing its rows.
+
+
 <!-- ---------------------
       v1.9.0
      --------------------- -->
@@ -53,8 +115,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - The project now uses the GNU General Public License (GPL) v3; license file added
-- Introduces new Python package for storing and loading numpy arrays; in can be installed with `pip install gputils-api`; 
-  unit tests and documentation
+- Python package for storing and loading NumPy arrays, with tests and documentation; install with `pip install gputils-api`.
 
 ### Fixed
 
@@ -70,8 +131,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-- Method `saveToFile` becomes `DTensor<T>::saveToFile(std::string pathToFile, Serialisation ser)`, i.e., the user can 
-  choose whether to save the file as a text (ASCII) file, or a binary one
+- `DTensor<T>::saveToFile(std::string pathToFile, Serialisation ser)` supports text (ASCII) and binary output.
 
 ### Changed
 
@@ -115,12 +175,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
-- (Breaking change) The methods `CholeskyFactoriser::factorise`, `CholeskyFactoriser::solve`, `QRFactoriser::factorise`,
-  `QRFactoriser::leastSquares`, and `QRFactoriser::getQR` are now `void` and do not
-  return a status code. Instead, a status code is returned by called `statusCode`.
-  This change leads to a reduction in data being downloaded from the GPU.
-- In `Svd` a status code (`bool`) is returned from `Svd<double>::factorise` only if the
-  `#GPUTILS_DEBUG_MODE` is defined, otherwise, the method returns always `true`.
+- **Breaking:** Cholesky factorisation/solve and QR factorisation/least-squares/extraction return `void`; query status separately to reduce GPU downloads.
+- `Svd<double>::factorise` returns a status only with `GPUTILS_DEBUG_MODE`; otherwise it always returns `true`.
 - New base class `IStatus` used for a universal implementation of `info()`
 
 
@@ -142,7 +198,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
-- Memory management improvements: we got rid of `pointerToMatrices`, which would unnecessarily allocate memory and `addAB` does not allocate any new memory internally.
+- Remove unnecessary `pointerToMatrices` allocations; `addAB` no longer allocates memory internally.
 
 <!-- ---------------------
       v1.3.0
@@ -217,7 +273,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-- Implementation of `DTensor<T>`, which is our basic entity for data storage and maniputation (supports basic linear algebra using `cublas` and `cusolver`); implementation of `=+`, `-=`, `*=` (for scalars and other tensors), `+`, `-`, `*` (scalars and tensors), printing (using `std::cout <<`), computation of norms (Frobenius and sum of absolute values of all elements); device vectors and matrices are tensors  
+- `DTensor<T>` provides GPU tensor storage, arithmetic, printing, and norms using cuBLAS and cuSolver; includes device vectors and matrices.
 - Singular value decomposition using `cublas`
 - Least-squares on tensors
 - Computation of nullspace matrices (on tensor objects)

CMakeLists.txt

@@ -67,6 +67,9 @@ if(NOT GPUTILS_BUILD_TEST)
     set(GPUTILS_BUILD_TEST OFF)  # Set to ON for local testing (or add `-DGPUTILS_BUILD_TEST=ON` to your CMake profile)
 endif()
 if (GPUTILS_BUILD_TEST)
+    # Register tests in the top-level build tree so `ctest --test-dir build`
+    # discovers tests added by the test subdirectory.
+    enable_testing()
     add_subdirectory(test)
 endif()
 unset(GPUTILS_BUILD_TEST CACHE)