own: lazy fills through @nodrops C calls, stores then used, a let given fresh values on some paths (TSLANG 63-66)

TSLANG_ISSUES.md

@@ -435,6 +435,50 @@ array spec's ruling T4-D accepts for a parameter. In the sweep, 4 errors less (8
 keep others. Regression tests `tslang/test/tester/tests/00tslang_issue_62.ts` (every model) and
 `own/own_err_borrowed_array_pushed_returned.ts`.
 
+63 (a lazy getter that fills an empty field with a C call in between: `if (!this.cachedVertexBuffer)
+this.cachedVertexBuffer = new NriBuffer(Nri_GetGltfMeshVertexBuffer(this.handle));` in `core/nri.ts`, which
+made every call of such a getter one that "may release or overwrite" whatever the caller borrowed) is fixed.
+A fill of an empty field (TSLANG 47) may not run a call that can reach the field in between, and an imported
+C function could reach anything. One declared `@nodrops`, with no export, import or dllimport, not variadic,
+and given and returning only numbers, strings, `null`/`undefined` or `@cpp` handles (or unions of them), runs
+no function of this module and gets nothing that leads to one: the fill now holds across it. Regression test
+`tslang/test/tester/tests/00tslang_issue_63.ts` (every model; it also holds 64 and 65).
+
+64 (a value moved into a field of an object already pushed into an array, then used: `const queuedFrame =
+new QueuedFrame(); this.queuedFrames.push(queuedFrame); queuedFrame.pbrView = pbrView; if (!pbrView) ...` in
+`examples/nri/simple_pbr_nri.ts`, refused "is used here after its value was moved") is fixed. TSLANG 50's
+borrow of the field the value went into is now also taken after the push, and for a value built by `new`
+in one arm of a `?:` (its construction's own uses are not uses of the merge). Popping or splicing the
+object out before the use, or overwriting the field, is still refused. Regression test
+`own/own_err_pushed_object_field_popped.ts`.
+
+65 (a value moved into an element, or a local's value moved into a field or an element, then used: `let
+pipeline = null; if (loaded) pipeline = create(); this.baselinePipelines[logicOp] = pipeline; return
+pipeline;` in `examples/nri/logic_op_dynamic_state_nri.ts`) is fixed. The use borrows the element (or field)
+from the store on, as TSLANG 50 does for a fresh value, when the local's slot is not assigned again after the
+store and the store does not run again (a loop). The element reset or popped before the use, or the store in
+a loop, is still refused. Regression test `own/own_err_local_stored_element_reset.ts`.
+
+With 63-65 in, errors in the sweep went from 807 to 767 (21 sources have fewer; `simple_pbr_nri.ts` 6 to 1),
+and no source has more; 119 sources still fail, each for other errors as well.
+
+66 (a `let` borrowing a field, then given a fresh value on some paths: `let dir = this.dir; ... if (yaw != 0.0
+|| pitch != 0.0) { dir = rotateAround(...); } ... this.dir = normalize(dir);` in
+`core/donut_first_person_camera.ts`'s `animate`, refused "borrows a field and cannot be assigned; -mm=own
+cannot prove that yet") is fixed. The `let` would own its value on one path and borrow it on the other, and
+-mm=own has no flag to release it on one only. Where the dry run finds such a `let` (initialized from a
+place, only read, assigned and released), each fresh value it is given goes into a slot of its own, released
+where the `let` is (an assignment of another fresh value, the scope's end, an unwind), and the `let` is given
+a borrow of that slot's value; it then holds only borrows, as TSLANG 56's do. Each borrow of the slot is
+checked like any other: it may not be stored, returned or captured, nor used after the slot's value is
+released. The `let` given its field on the other paths stays bounded by the field, so the field overwritten
+before a use is still refused. In the sweep, the camera module compiles, and with its facts the examples that
+call it do too: errors went from 767 to 664, and failing sources from 119 to 118 (`async_compute_bloom_nri.ts`,
+`multi_draw_indirect_nri.ts` and `terrain_tessellation_nri.ts` 8 errors to 1, both `ray_tracing_*_nri.ts`
+ports 25 to 6), and no source has more. Regression tests `tslang/test/tester/tests/00tslang_issue_66.ts`
+(every model; loops, a class-typed `let`, a throw), `own/own_err_split_let_stored.ts` and
+`own/own_err_split_let_field_overwritten.ts`.
+
 For native_ts_graphics, from the same sweep (its code, not the compiler):
 - `core/ray_tracing_extended_scene.ts`: the exported classes' constructors keep their array parameters
   (`this.origin = origin`), which an exported constructor has to say: `@keeps("origin", "direction")`
@@ -443,9 +487,9 @@ For native_ts_graphics, from the same sweep (its code, not the compiler):
 - `examples/dynamic_primitive_clipping.ts:223`, `desc.setDepthClip(clip)` with `clip` the loop's `int`,
   fails under every model since #846 (a number is not a `boolean` argument, TS2345). alpha127 still
   takes it; the next release won't. `desc.setDepthClip(clip != 0)` fixes it.
-- Under `-mm=own`, 4 `core/` modules do not compile yet (`donut_first_person_camera.ts`,
-  `nri_donut_textures.ts`, `ray_tracing_extended_scene.ts`, `rtxpt_game.ts`), and nor do many examples,
-  which #310 did not port: with every facts object built and 50-62 in, 119 of the 276 sources fail. The
-  largest group left is "borrows a field but is used here after it may be released or overwritten" (about
-  200 sites), then "used here after its value was moved" (37: mostly a value a pushed object or an
+- Under `-mm=own`, 3 `core/` modules do not compile yet (`nri_donut_textures.ts`,
+  `ray_tracing_extended_scene.ts`, `rtxpt_game.ts`), and nor do many examples,
+  which #310 did not port: with every facts object built and 50-66 in, 118 of the 276 sources fail (664 errors). The
+  largest group left is "borrows a field but is used here after it may be released or overwritten" (396,
+  93 of them `this.sample.run` callbacks), then "used here after its value was moved" (58: mostly a value a pushed object or an
   earlier call keeps, then returned or used).

tslang/include/TypeScript/Defines.h

@@ -56,6 +56,8 @@
 // `ts.RetainSlot` to pair the release with. The ownership verifier reads this attribute as the
 // retain it stands in for.
 #define OWNED_LOCAL_CONSUMED_ATTR_NAME "__owned_consumed"
+// the owning slot that holds the fresh values a `let` borrowing a field is given (TSLANG 66)
+#define OWN_LET_HOLDER_ATTR_NAME "__own_let_holder"
 
 // Marks an OWNED_RESULT_ATTR_NAME operation whose reference some receiver has taken over, so the
 // +1 it produced is now somebody's to give back. Set at each of the receiving sites (§9.25) and

tslang/lib/TypeScript/OwnershipFacts.h

@@ -647,10 +647,32 @@ inline bool isFreshMerge(mlir::BlockArgument argument, unsigned depth = 0)
             continue;
         }
 
-        // the fresh value and the views it reaches the branch through, used by nothing else
+        // the fresh value and the views it reaches the branch through, used by nothing else - but
+        // what makes an object `ts.New` gives (TSLANG 64: `c ? new V(i) : null`): its vtable's store
+        // into field 0 and the call of its constructor
+        auto constructs = [](mlir::Value value, mlir::Operation *user) {
+            if (!value.getDefiningOp<mlir_ts::NewOp>())
+            {
+                return false;
+            }
+
+            if (auto propertyRefOp = mlir::dyn_cast<mlir_ts::PropertyRefOp>(user))
+            {
+                return propertyRefOp.getPosition() == 0 && llvm::all_of(propertyRefOp->getUsers(), [&](mlir::Operation *fieldUser) {
+                           auto storeOp = mlir::dyn_cast<mlir_ts::StoreOp>(fieldUser);
+                           return storeOp && storeOp.getReference() == propertyRefOp.getResult();
+                       });
+            }
+
+            auto callOp = mlir::dyn_cast<mlir_ts::SymbolCallInternalOp>(user);
+            return callOp && callOp->getNumOperands() > 0 && callOp->getOperand(0) == value &&
+                   callOp.getCallee().ends_with(".constructor") &&
+                   llvm::count(callOp->getOperands(), value) == 1;
+        };
         for (auto value = incoming;;)
         {
-            if (!value.hasOneUse())
+            auto uses = llvm::count_if(value.getUsers(), [&](mlir::Operation *user) { return !constructs(value, user); });
+            if (uses != 1)
             {
                 return false;
             }

tslang/lib/TypeScript/OwnershipInferenceBorrowers.cpp

@@ -249,17 +249,21 @@ namespace own_inference
         return true;
     }
 
+    bool OwnershipInferencePass::letBorrows(mlir::Value value)
+    {
+        // a read of a place or of a parameter - or of a split `let`'s holder (TSLANG 66)
+        auto root = rootOf(value);
+        auto rootLoad = root.getDefiningOp<mlir_ts::LoadOp>();
+        auto slot = rootLoad ? rootLoad.getReference().getDefiningOp<mlir_ts::VariableOp>() : mlir_ts::VariableOp();
+        int index = -1;
+        return placeReadOf(root) || (slot && (isParameterSlot(slot, index) || slot->hasAttr(OWN_LET_HOLDER_ATTR_NAME)));
+    }
+
     bool OwnershipInferencePass::borrowsOnly(mlir_ts::VariableOp varOp)
     {
         // `let layout = this.computeLayout; if (fp16) layout = this.computeLayoutFP16;` (TSLANG 56):
         // an owning `let` given only reads of places and parameters, each a borrow of its own
-        auto borrowed = [&](mlir::Value value) {
-            auto root = rootOf(value);
-            auto rootLoad = root.getDefiningOp<mlir_ts::LoadOp>();
-            auto slot = rootLoad ? rootLoad.getReference().getDefiningOp<mlir_ts::VariableOp>() : mlir_ts::VariableOp();
-            int index = -1;
-            return placeReadOf(root) || (slot && isParameterSlot(slot, index));
-        };
+        auto borrowed = [&](mlir::Value value) { return letBorrows(value); };
 
         if (!varOp.getInitializer() || !isOwningVariable(varOp) || varOp.getCaptured().value_or(false) ||
             !borrowed(varOp.getInitializer()))
@@ -390,6 +394,155 @@ namespace own_inference
         return nullptr;
     }
 
+    void OwnershipInferencePass::splitBorrowingLets()
+    {
+        // `let dir = this.dir; if (turn) dir = rotate(dir); this.dir = normalize(dir);` (TSLANG 66): each
+        // fresh value the `let` is given goes into a holder, an owning slot of its own released where the
+        // `let` is, and the `let` borrows it from the holder - as it borrows the field
+        for (auto *candidate : splitLetCandidates)
+        {
+            auto varOp = mlir::cast<mlir_ts::VariableOp>(candidate);
+            auto slot = varOp.getResult();
+            if (!varOp.getInitializer() || !letBorrows(varOp.getInitializer()) || varOp.getCaptured().value_or(false))
+            {
+                continue;
+            }
+
+            llvm::SmallVector<mlir_ts::StoreOp> fresh;
+            llvm::SmallVector<mlir::Operation *> exits;
+            auto splits = true;
+            for (auto *user : slot.getUsers())
+            {
+                if (mlir::isa<mlir_ts::LoadOp, mlir_ts::RetainSlotOp, mlir_ts::DebugVariableOp>(user))
+                {
+                    continue;
+                }
+
+                // the release of its old value before an assignment, or of its last at a way out
+                if (mlir::isa<mlir_ts::ReleaseSlotOp>(user))
+                {
+                    auto next = mlir::dyn_cast_or_null<mlir_ts::StoreOp>(user->getNextNode());
+                    if (!next || next.getReference() != slot)
+                    {
+                        exits.push_back(user);
+                    }
+
+                    continue;
+                }
+
+                auto storeOp = mlir::dyn_cast<mlir_ts::StoreOp>(user);
+                if (!storeOp || storeOp.getReference() != slot)
+                {
+                    splits = false;
+                    break;
+                }
+
+                if (letBorrows(storeOp.getValue()))
+                {
+                    continue;
+                }
+
+                auto release = mlir::dyn_cast_or_null<mlir_ts::ReleaseSlotOp>(storeOp->getPrevNode());
+                if (!release || release->getOperand(0) != slot)
+                {
+                    splits = false;
+                    break;
+                }
+
+                fresh.push_back(storeOp);
+            }
+
+            if (!splits || fresh.empty())
+            {
+                continue;
+            }
+
+            auto type = mlir::cast<mlir_ts::RefType>(slot.getType()).getElementType();
+            mlir::OpBuilder builder(varOp);
+            builder.setInsertionPointAfter(varOp);
+            auto empty = builder.create<mlir_ts::DefaultOp>(varOp.getLoc(), type);
+            auto holder = builder.create<mlir_ts::VariableOp>(varOp.getLoc(), slot.getType(), empty, varOp.getCapturedAttr(),
+                                                              mlir::IntegerAttr());
+            holder->setAttr(OWNED_LOCAL_ATTR_NAME, builder.getUnitAttr());
+            holder->setAttr(OWN_LET_HOLDER_ATTR_NAME, builder.getUnitAttr());
+            for (auto storeOp : fresh)
+            {
+                storeOp->getPrevNode()->setOperand(0, holder.getResult());
+                storeOp.getReferenceMutable().assign(holder.getResult());
+                builder.setInsertionPointAfter(storeOp);
+                auto read = builder.create<mlir_ts::LoadOp>(storeOp.getLoc(), type, holder.getResult());
+                builder.create<mlir_ts::StoreOp>(storeOp.getLoc(), read, slot);
+            }
+
+            for (auto *exit : exits)
+            {
+                builder.setInsertionPointAfter(exit);
+                builder.create<mlir_ts::ReleaseSlotOp>(exit->getLoc(), holder.getResult());
+            }
+        }
+    }
+
+    void OwnershipInferencePass::checkLetHolders(llvm::SetVector<mlir::Operation *> &toErase)
+    {
+        getFunction()->walk([&](mlir_ts::VariableOp holder) {
+            if (!holder->hasAttr(OWN_LET_HOLDER_ATTR_NAME))
+            {
+                return;
+            }
+
+            auto ends = slotEnds(holder.getResult());
+            for (auto *user : holder.getResult().getUsers())
+            {
+                auto readOp = mlir::dyn_cast<mlir_ts::LoadOp>(user);
+                if (!readOp || !readOp->hasOneUse())
+                {
+                    continue;
+                }
+
+                // the `let` given the holder's value: its reads up to its next assignment borrow it
+                auto storeOp = mlir::dyn_cast<mlir_ts::StoreOp>(*readOp->getUsers().begin());
+                auto borrower = storeOp ? storeOp.getReference().getDefiningOp<mlir_ts::VariableOp>() : mlir_ts::VariableOp();
+                if (!borrower || storeOp.getValue() != readOp.getResult())
+                {
+                    continue;
+                }
+
+                auto name = varName(borrower);
+                llvm::SmallVector<BorrowUse> uses;
+                llvm::SmallVector<mlir::Operation *> bookkeeping;
+                // (a `let` declared in a loop: its declaration starts it afresh)
+                mlir::Operation *declaration = borrower;
+                if (auto *kept = collectBorrowerUses(borrower, declaration, uses, bookkeeping, storeOp))
+                {
+                    if (isAssignmentOf(kept, borrower))
+                    {
+                        reportBorrowerAssigned(kept, name, name);
+                    }
+                    else
+                    {
+                        reportBorrowEscapes(kept, name, name);
+                    }
+
+                    return;
+                }
+
+                for (auto *end : ends)
+                {
+                    for (auto &use : uses)
+                    {
+                        if (reachableAfter(end, use.op, use.kills))
+                        {
+                            reportBorrowOutlives(use.op, end, name, name);
+                            return;
+                        }
+                    }
+                }
+
+                toErase.insert(bookkeeping.begin(), bookkeeping.end());
+            }
+        });
+    }
+
     bool OwnershipInferencePass::isAssignmentOf(mlir::Operation *op, mlir_ts::VariableOp varOp)
     {
         auto storeOp = mlir::dyn_cast<mlir_ts::StoreOp>(op);

tslang/lib/TypeScript/OwnershipInferencePass.cpp

@@ -38,8 +38,11 @@ namespace own_inference
             if (!passed)
             {
                 makeStringCopies(toErase);
+                splitBorrowingLets();
             }
 
+            splitLetCandidates.clear();
+
             for (auto &[op, name] : dryRunAttrs)
             {
                 op->removeAttr(name);
@@ -352,6 +355,7 @@ namespace own_inference
         checkCannotShare();
         checkBorrowingRecords();
         checkRecordResults();
+        checkLetHolders(toErase);
         toErase.insert(fieldBorrowReleases.begin(), fieldBorrowReleases.end());
 
         if (dryRun)

tslang/lib/TypeScript/OwnershipInferencePass.h

@@ -174,6 +174,15 @@ class OwnershipInferencePass
     // Strings as values (spec 22.3): each string retain the dry run did not erase is rewritten, where
     // its one use can be found, into a copy for that use.
     void makeStringCopies(const llvm::SetVector<mlir::Operation *> &erased);
+    // `let dir = this.dir; if (turn) dir = rotate(dir); this.dir = normalize(dir);` (TSLANG 66): a `let`
+    // borrowing a field, then given a fresh value, the dry run found it could not be assigned. The fresh
+    // values go into an owning slot of their own (its holder), released where the `let` is, and the
+    // `let` borrows the holder's value: it then holds only borrows (borrowsOnly).
+    llvm::SetVector<mlir::Operation *> splitLetCandidates;
+    void splitBorrowingLets();
+    // The checks of a split `let`'s borrows of its holder's values: none outlives the holder's value
+    // nor escapes.
+    void checkLetHolders(llvm::SetVector<mlir::Operation *> &toErase);
 
     // The value a parameter of this function is, when its callers were told this function keeps it
     // or returns a borrow of it (spec 15): copying it would leak what the caller gave up.
@@ -295,6 +304,8 @@ class OwnershipInferencePass
     // An owning `let` given only reads of places and parameters, and assigned at least once
     // (TSLANG 56): each value it is given is a borrow until the next assignment.
     bool borrowsOnly(mlir_ts::VariableOp varOp);
+    // Is `value` a read a `let` given it only borrows (borrowsOnly)?
+    bool letBorrows(mlir::Value value);
     // The `let` (borrowsOnly) that `user` assigns `value` to - the store itself, or rc's retain right
     // before it - or null.
     mlir_ts::VariableOp borrowingAssignmentOf(mlir::Operation *user, mlir::Value value);

tslang/lib/TypeScript/OwnershipInferencePlaceReads.cpp

@@ -973,6 +973,12 @@ namespace own_inference
 
             if (!kept && isAssignmentOf(letKept, letKeeps))
             {
+                // a candidate for a holder of its fresh values (TSLANG 66)
+                if (dryRun)
+                {
+                    splitLetCandidates.insert(letKeeps);
+                }
+
                 reportPlaceBorrowerAssigned(letKept, varName(letKeeps), place);
             }
             else