Place traits #3921
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,257 @@ | ||
| - Feature Name: `place_traits` | ||
| - Start Date: 2026-01-23 | ||
| - RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/3921) | ||
| - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) | ||
|
|
||
| ## Summary | ||
| [summary]: #summary | ||
|
|
||
| This RFC introduces the `Place` trait. This trait allows arbitrary types to implement the | ||
| special derefence behavior of the `Box` type. In particular, it allows an arbitrary type | ||
| to act as an owned place allowing values to be (partially) moved out and moved back in | ||
| again. | ||
|
|
||
| ## Motivation | ||
| [motivation]: #motivation | ||
|
|
||
| Currently the Box type is uniquely special in the rust ecosystem. It is unique in acting | ||
| like an owned variable, and allows a number of special optimizations for direct | ||
| instantiation of other types in the storage allocated by it. | ||
|
|
||
| This special status comes with two challenges. First of all, Box gets its special status | ||
| by being deeply interwoven with the compiler. This is somewhat problematic as it requires | ||
| exactly matched definitions of how the type looks between various parts of the compiler | ||
| and the standard library. Moving box over to a place trait would provide a more pleasant | ||
| and straightforward interface between the compiler and the box type, at least in regards | ||
| to the move behavior. | ||
|
|
||
| Second, it is currently impossible to provide a safe interface for user-defined smart | ||
| pointer types which provide the option of moving data in and out of it. There have been | ||
| identified a number of places where such functionality could be interesting, such as when | ||
| removing values from containers, or when building custom smart pointer types for example | ||
| in the context of an implementation of garbage collection. | ||
|
|
||
| ## Guide-level explanation | ||
| [guide-level-explanation]: #guide-level-explanation | ||
|
|
||
| This proposal introduces a new unsafe trait `Place`: | ||
| ```rust | ||
| unsafe trait Place: DerefMut { | ||
| fn place(&mut self) -> *mut Self::Target | ||
|
Contributor
There was a problem hiding this comment. So, it's a little strange why this method is necessary when it appears that any implementation would just put a call to
Author
There was a problem hiding this comment. Best I understand the semantics of mutable references, it would be unsound to have one to a value that is uninitialized. And the semantics of moving in and out of the
Contributor
There was a problem hiding this comment. Hmm, but wouldn't that mean that you have to take
Author
There was a problem hiding this comment. Ok, I answered more completely in the other thread, see #3921 (comment). It basically comes down to that the argument to place is the Box-Like, which is (and should be, otherwise use of this trait is going to be a nightmare) still initialized, even though its Contents might not be. The non-initialized status of the Contents rules out the use of references for that, hence the extra function and the pointer. |
||
| } | ||
| ``` | ||
|
|
||
| The `Place` trait essentially allows values of the type to be treated as an already- | ||
| existing box. That is, they behave like a variable of the type `Deref::Target`, just | ||
| stored in a different location than the stack. This means that values of type | ||
| `Deref::Target` can be (partially) moved in and out of dereferences of the type, with the | ||
| borrow checker ensuring soundness of the resulting code. As an example, if `Foo` | ||
| implements `Place` for type `Bar`, the following would become valid rust code: | ||
| ```rust | ||
| fn baz(mut x: Foo) -> Foo { | ||
| let y = *x; | ||
| *x = y.destructive_update() | ||
| x | ||
| } | ||
| ``` | ||
|
|
||
| When implementing this trait, the type itself effectively transfers some of the responsibilities for managing the value behind the pointer returned by `Place::place`, also called the content, to the compiler. In particular, the type itself should no longer count on the ccontent being properly initialized and dropable when its `Drop` implementation or `Place::place` implementation is called. However, the compiler still guarantees that, as long as the type implementing the place is always created with a value in it, and that value is never removed through a different mechanism than dereferencing the type, all other calls to member functions can assume the value to be implemented. | ||
This comment was marked as resolved.
Sorry, something went wrong.
Author
There was a problem hiding this comment. Just to clarify for those following along, yes the intent is that the compiler will always emit the relevant drop code for the contents, and the Place never has to drop the contents. |
||
|
|
||
| In general, the compilers requirements are met when | ||
| - The pointer returned by `place` should be safe to mutate through, and should be live | ||
| for the lifetime of the mutable reference to `self` passed to `Place::place`. | ||
| - On consecutive calls to `Place::place`, the status of whether the content is initialized should not be changed. | ||
| - Drop must not drop the contents, only the storage for it. | ||
|
There was a problem hiding this comment. If There was a problem hiding this comment. Here's an idea: Have an additional trait: unsafe trait DropShell: Drop + Place {
fn drop_shell(&mut self);
}If a type implements Implementing
Author
There was a problem hiding this comment. So how this currently works for Box is that the compiler is always responsible for dropping the contents. The suggestion here is to use the same contract for places.
Member
There was a problem hiding this comment. Fwiw we have this trait (we call it
Author
There was a problem hiding this comment. Just a short note, the choice for not using a separate trait in this proposal is deliberate. This proposal deliberately tries to simplify the compiler implementation as much as possible to reduce risk there. Always having the compiler drop the contents simplifies both the compiler implementation and the operational semantics. And in such a case, I didn't see a need for a separate trait for dropping the container without its contents, as that is always what happens. |
||
| - Newly initialized values of the type implementing `Place` must have their content initialized. | ||
|
|
||
| There is one oddity in the behavior of types implementing `Place` to be aware of. | ||
| Automatically elaborated dereferences of values of such types will always trigger an abort | ||
| on panic, instead of unwinding when that is enabled. However, generic types constrained to | ||
| only implement Deref or DerefMut but not Place will always unwind on panics during | ||
| dereferencing, even if the underlying type also implements Place. | ||
|
|
||
| ## Reference-level explanation | ||
| [reference-level-explanation]: #reference-level-explanation | ||
|
|
||
| This proposal introduces one new main language item, the traits `Place`. We also introduce a number of secondary language items which are used to make implementation easier and more robust, which we shall define as they come up below. | ||
|
|
||
| A type implementing the trait Place is required to act as a place for borrow checking. Throughout the rest of this text, the contents of the memory pointed at by the pointer returned by the `Place::place` function shall be refered to as the content of the place. For a type to satisfy the above requirement, its implementation must in particular guarantee that | ||
| - Safe code shall not modify the initialization status of the contents. | ||
| - Unsafe code shall preserve the initialization status of the contents between two derefences of teh type's values. | ||
| - Values of the place type for which the content is uninitialized shall not be able to be created in safe code. | ||
|
Contributor
There was a problem hiding this comment. This is… a very confusing set of safety requirements considering how they're effectively already guaranteed by the intrinsic safety requirements of the language: you can't de-initialize the contents of something with a mutable reference, and unsafe code is expected to uphold this regardless of whether the reference is converted into a pointer or not.
Author
There was a problem hiding this comment. I agree that the formulation here is less than ideal, however there is something real that is asked here, that is unfortunately not guaranteed by the borrow checker alone. For example, if somebody defines the type struct InplaceBox<T>(MaybeUninit<T>);
impl<T> Deref for InplaceBox<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
unsafe { self.0.assume_init_ref() }
}
}
impl<T> DerefMut for InplaceBox<T> {
fn deref_mut(&mut self) -> &mut Self::Target {
unsafe { self.0.assume_init_mut() }
}
}
unsafe impl<T> Place for InplaceBox<T> {
type NewArg = ();
fn place(&mut self) -> *mut Self::Target {
self.0.as_mut_ptr()
}
}Then as part of the unsafe contract for place they promise to for example not to do stuff like #[derive(Debug, Clone, Copy)]
enum NonZero { One = 1, Two = 2 }
pub fn foo() {
let ipb = InplaceBox(MaybeUninit::init(NonZero::One))
println!("{:?}", *ipb); // Still OK, ipb contains something
ipb.0 = MaybeUninit:zeroed(); // Should be forbidden, because borrow checker should still think following is ok.
println!("{:?}", *ipb); // UB happens here now, since even though the borrow checker thinks this is fine, it is not, because of the line above.
}Forbidding these sorts of shenanigans is what I am trying to capture with these requirements. If you have suggestions for how to better formulate that I'd love those.
Contributor
There was a problem hiding this comment. I guess that, as I also pointed out in the other comment thread, it's not really clear how this UB is possible based upon what the API is trying to achieve. Yeah, that example is obviously UB, but it's unclear how the Like, no matter what, safe code should not be allowed to create an invalid value, and if you're specifically moving a value out of a place, you kind of necessitate that the container be dropped in some way as part of this process, so, further references will not be possible. For There was a problem hiding this comment. Possibly the trait could remain unstable itself while having a pointer-field based compiler generated implementation path, similar to
Member
There was a problem hiding this comment.
You can currently move out of a fn main() {
let mut a = Box::new(Box::new(vec![0]));
drop(**a); // Deinitialize the inner box without moving it.
**a = vec![]; // Reinitialize the inner box without moving it.
}There was a problem hiding this comment. I do disagree. You'd be writing uninit bytes into a (stack-)allocation of type As for how I think of it, the major novelty of the place API is manipulating information about places, not types. Using traits for more than markers is odd to me since those would be type-properties and algorithms—neither of which express information about places. That invites an unrelated layer of complexity around a new IR operation. The unaddressed question of what happens internally (expressed in MIR maybe) seems more pressing—then we can check how the preconditions for those can be encoded into traits / other interfaces.
Author
There was a problem hiding this comment. Apologies, perhaps having the storage on the stack was an unwise choice on my part. Where would you articulate the unsoundness being in this example: Assuming that the particular malloc is infallible and does not violate alignment. To me, this is no different than the previous example, but this time there is no requirement on the storage that it be a valid value, so there are still coming requirements from the fact that MyKindOfBox is Boxlike, and therefore does want specific things from its (exposed!) backing storage. I dont think we can avoid specifying those assumptions just because we can't explicitly implement the trait that makes it a Boxlike. There was a problem hiding this comment. I'm still not entirely sure what you're asking. The trait moves responsibility for the place to the function body; you'd also required that all values have had their places initialized (following the section at 'requires are met …') and from that point on the storage has to be valid for The RFC is missing an operational semantics of how the pointer is invalidated for other writes in a way that the compiler can actually do something with the information but the moment you implement the trait and create a value of it you opt-in to not allow the write. How to disallow it seems to be the responsibility of the author of the type, here by privacy and not doing a raw write itself. You were asking how the derive makes this clearer. Well first of all we can then actually talk about operation semantics. By identifying the pointer as a field instead of a method we have an actual pointer value to manipulate (and invalidate), not an ephemeral derived copy. Secondly, we can say that construction does something with that value when it is assigned as a field (which makes construction come with a precondition, a valid pointer, so it must be usually When you justify Edit: having operational semantics of struct-expressions invalidate the pointer is just an example to demonstrate the difference between the two approaches; I think it's not entirely what we'd want. Alternatively, and stronger, we might even require the pointer validity as part of the representational invariants of the macro-annotated type. This would allow dereferentiablity assumptions recursively through other references / … (with the decision if we want recursive representation invariants still being outstanding). Or something in between. As long as we have a well-defined notion of the raw pointer / pointee place inside the custom place there should be ample options.
Author
There was a problem hiding this comment. Ok, I see what you are getting at, and focussing more on the operational semantics of the contents is likely a good direction for making the explanation in this area clearer. I have some time for rewrites on friday and will try to rework this section to clarify that. I am not yet convinced that a derive macro yields enough value to make up for the extra implementation complexity and reduced utility, but that is probably better evaluated once I have a new version of the requirements language here.
Author
There was a problem hiding this comment. Ok, I have done the promised rewrite, and I at least feel much happier with the new section on requirements and provided guarantees by the compiler in the rfc. It can be found on lines 85-102, and I suggest we continue the discussion in a review comment on those lines. |
||
| In the above context, the contents is also considered uniitialized if the whole or parts of the value of the contents has been moved out, or a destructor has been called upon them. | ||
|
|
||
| Dereferences of a type implementing `Place` can therefore be lowered directly to MIR, only | ||
| being elaborated in a pass after borrow checking. This allows the borrow checker to fully | ||
| check that the moves of data into and out of the type are valid. | ||
|
|
||
| The dereferences and drops of the contained value can then be elaborated in the passes | ||
| after borrow checking. This process will be somewhat similar to what is already done for | ||
| Box, with the difference that dereferences of types implementing `Place` may panic. We | ||
| propose to handle these panics by aborting to avoid introducing interactions with drop | ||
| elaboration and new execution paths not checked by the borrow checker. | ||
|
|
||
| In order to generate the function calls to the `Place::place` and `Deref::deref` during | ||
| the dereference elaboration we propose making these functions additional language items. | ||
|
|
||
| ## Drawbacks | ||
| [drawbacks]: #drawbacks | ||
|
|
||
| There are three main drawbacks to the design as outlined above. First, the traits are | ||
| unsafe and come with quite an extensive list of requirements on the implementing type. | ||
| This makes them relatively tricky and risky to implement, as breaking the requirements | ||
| could result in undefined behavior that is difficult to find. | ||
|
|
||
| Second, with the current design the underlying type is no longer aware of whether or not | ||
| the space it has allocated for the value is populated or not. This inhibits functionality | ||
| which would use this information on drop to automate removal from a container. Note | ||
| however that such usecases can use a workaround with the user explicitly requesting | ||
| removal before being able to move out of a smart-pointer like type. | ||
|
|
||
| Finally, the type does not have runtime-awareness of when the value is exactly added. | ||
| This means that the proposed traits are not suitable for providing transparent locking of | ||
| shared variables to end user code. | ||
|
|
||
| In past proposals for similar traits it has also been noted that AutoDeref is complicated | ||
| and poorly understood by most users. It could therefore be considered problematic that | ||
| AutoDeref behavior is extended. However the behavior here is identical to what Box already | ||
| has, which is considered acceptable in its current state. | ||
|
|
||
| ## Rationale and alternatives | ||
| [rationale-and-alternatives]: #rationale-and-alternatives | ||
|
|
||
| Ideas for something like the `Place` trait design here can be found in past discussions of | ||
| DerefMove traits and move references. The desire for some way of doing move derefences | ||
| goes back to at least https://github.com/rust-lang/rfcs/issues/997. | ||
|
|
||
| The rationale behind the current design is that it explicitly sticks very closely to what | ||
| is already implemented for Boxes, which in turn closely mirror what can be done with stack | ||
| variables directly. This provides a relatively straightforward mental model for the user, | ||
| and significantly reduces the risk that the proposed design runs into issues in the | ||
| implementation phase. | ||
|
|
||
| ### DerefMove trait | ||
|
|
||
| Designs based on a simpler DerefMove trait have been previously proposed in the unmerged | ||
| [RFC2439](https://github.com/rust-lang/rfcs/pull/2439) and an [internals forum thread](https://internals.rust-lang.org/t/derefmove-without-move-why-dont-we-have-it/19701). | ||
| These come down to a trait of the form | ||
| ``` | ||
| trait DerefMove : DerefMut { | ||
| fn deref_move(self) -> Self::Target | ||
| } | ||
| ``` | ||
|
|
||
| The disadvantage of an approach like this is that it is somewhat unclear how to deal with | ||
| partial moves. This has in the past stopped such proposals in their tracks. | ||
|
|
||
| Furthermore, such a trait does not by itself cover the entirety of the functionality | ||
| offered by Box, and given its consuming nature it is unclear how to extend it. This also | ||
| leads to the potential for backwards incompatible changes to the current behavior of Box, | ||
| as has previously been identified. | ||
|
|
||
| ### &move based solutions | ||
|
|
||
| A separate class of solutions has been proposed based on the idea of adding &move | ||
| references to the type system, where the reference owns the value, but not the allocation | ||
| behind the value. These were discussed in an [unsubmitted RFC by arielb1](https://github.com/arielb1/rfcs/blob/missing-derefs/text/0000-missing-derefs.md), | ||
| and an [internals forum thread](https://internals.rust-lang.org/t/pre-rfc-move-references/14511) | ||
|
|
||
| Drawbacks of this approach have been indicated as the significant extra complexity which | ||
| is added to the type system with the extra type of reference. There further seems to be a | ||
| need for subtypes of move references to ensure values are moved in or out before dropping | ||
| to properly keep the allocation initialized or deinitialized after use as needed. | ||
|
|
||
| This additional complexity leads to a lot more moving parts in this approach, which | ||
| although the result has the potential to allow a bit more flexibility makes them less | ||
| attractive on the whole. | ||
|
|
||
| ### More complicated place traits | ||
|
|
||
| Several more complicated `Place` traits have been proposed by tema2 in two threads on the | ||
| internals forum: | ||
| - [DerefMove without `&move` refs](https://internals.rust-lang.org/t/derefmove-without-move-refs/17575) | ||
| - [`DerefMove` as two separate traits](https://internals.rust-lang.org/t/derefmove-as-two-separate-traits/16031) | ||
|
|
||
| These traits aimed at providing more feedback with regards to the length of use of the | ||
| pointer returned by the `Place::place` method, and the status of the value in that | ||
| location after use. Such a design would open up more possible use cases, but at the cost | ||
| of significantly more complicated desugarings. | ||
|
|
||
| Furthermore, allowing actions based on whether a value is present or not in the place | ||
| would add additional complexity in understanding the control flow of the resulting binary. | ||
| This could make understanding uses of these traits significantly more difficult for end | ||
| users of types implement these traits. | ||
|
davidv1992 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### Limited macro based trait | ||
|
|
||
| Going the other way in terms of complexity, a `Place` trait with constraints on how the | ||
| projection to the actual location to be dereferenced was proposed in [another internals forum thread](https://internals.rust-lang.org/t/derefmove-without-move-references-aka-box-magic-for-user-types/19910). | ||
|
|
||
| This proposal effectively constrains the `Place::deref` method to only doing field | ||
| projections and other dereferences. The advantage of this is that such a trait has far | ||
| less severe safety implications, and by its nature cannot panic making its use more | ||
| predictable. | ||
|
|
||
| However, the restrictions require additional custom syntax for specifying the precise | ||
| process, which adds complexity to the language and makes the trait a bit of an outlier | ||
| compared to the `Deref` and `DerefMut` traits. | ||
|
|
||
| ### Existing library based solutions | ||
|
|
||
| The [moveit library](https://docs.rs/moveit/latest/moveit/index.html) provides similar | ||
| functionality in its `DerefMove` trait. However, this requires unsafe code on the part of | ||
| the end user of the trait, which makes them unattractive for developers wanting the | ||
| memory safety guarantees rust provides. | ||
|
|
||
| ## Prior art | ||
| [prior-art]: #prior-art | ||
|
|
||
| The behavior enabled by the trait proposed here is already implemented for the Box type, | ||
| which can be considered prime prior art. Experience with the Box type has shown that its | ||
| special behaviors have applications. | ||
|
|
||
| Beyond rust, there aren't really comparable features that map directly onto the proposal | ||
| here. C++ has the option for smart pointers through overloading dereferencing operators, | ||
| and implements move semantics through Move constructors and Move assignment operators. | ||
| However, moves in C++ require the moved-out of place to always remain containing a valid | ||
| value as there is no intrinsic language-level way of dealing with moved-out of places in | ||
| a special way. | ||
|
|
||
| In terms of implementability, a small experiment has been done implementing the deref | ||
| elaboration for an earlier version of this trait at | ||
| https://github.com/davidv1992/rust/tree/place-experiment. That implementation is | ||
| sufficiently far along to support running code using the Place trait, but does not yet | ||
| properly drop the internal value, instead leaking it. | ||
|
|
||
| ## Unresolved questions | ||
| [unresolved-questions]: #unresolved-questions | ||
|
|
||
| The current design would require the use of `Deref::deref` in desugaring non-moving | ||
| accesses to types implementing `Place`. However, it is currently unclear whether it is | ||
| sound to do so if the value has already been partially moved out. | ||
|
|
||
| Right now, the design states that panic in calls to `Deref::deref` or `Place::place` can | ||
| cause an abort when the call was generated in the MIR. This is done as it is at this point | ||
| somewhat unclear how to handle proper unwinding at the call sites for these functions. | ||
| However, it may turn out to be possible to implement this with proper unwinding, in which | ||
| case we may want to consider handling panics at these call sites the same as for ordinary | ||
| code. | ||
|
|
||
| ## Future possibilities | ||
| [future-possibilities]: #future-possibilities | ||
|
|
||
| Should the trait become stabilized, it may become interesting to implement non-copying | ||
| variants of the various pop functions on containers within the standard library. Such | ||
|
Comment on lines
+293
to
+294
Contributor
There was a problem hiding this comment. What would the signatures of these non-copying variants be like? I'm guessing you mean something looking like: impl<T> Vec<T> {
fn pop_in_place<'a>(&'a mut self) -> Option<impl DerefMove<Target = T> + use<'a, T>> {
self.set_len(self.len().checked_sub(1)?);
Some(OwnInUninit(&mut self.spare_capacity_mut()[0]))
}
}
// Safety: self.0 must point to a valid `T` when constructed
struct OwnInUninit<'a, T>(&'a mut MaybeUninit<T>);
unsafe impl<T> DerefMove for OwnInUninit<'_, T> {
/* type Target = T */
fn place(&self) -> *const Self::Target {
self.0.assume_init_ref()
}
fn place_mut(&mut self) -> *mut Self::Target {
self.0.assume_init_mut()
}
}Presuming I’m correct about this sketch, something interesting I notice is that There was a problem hiding this comment. The main difference from this to #1646 seems to me to be that latter is an operation to be written at any point whereas this work through the initialization itself. By not giving an independent choice of where the initialization of the smart pointer itself relative to the pointer to the interior is valid it can avoid some of the drop-order discussion. You could never run into the example in Impure DerefMove for instance. You also can't 'move' from a
Author
There was a problem hiding this comment. @kpreid The signatures you suggest are indeed correct. The implementation shouldn't use the assume functions but rather the pointer producing functions, but that is a minor detail that doesn't change the signature. On the complexity of move references, as I see it 197g's analysis is completely on point. This proposal has been written explicitly with the intent to keep the compiler side as simple as possible. That results in offloading some of the complexity to the implementers for more complicated use cases, such as having to manually track whether a container is still responsible for dropping contents in the scenario you outlined. |
||
| functions could allow significant optimizations when used in combination with large | ||
| elements in the container. | ||
|
|
||
| It may also be interesting at a future point to reconsider whether the unsized_fn_params | ||
| trait should remain internal, in particular once Unsized coercions become usable with user | ||
| defined types. However, this decision can be delayed to a later date as sufficiently many | ||
| interesting use cases are already available without it. | ||
|
|
||
| Finally, there is potential for the trait as presented here to become useful in the in | ||
| place initialization project. It could be a building block for generalizing things like | ||
| partial initialization to smart pointers. This would require future design around an api | ||
| for telling the borrow checker about new empty values implementing Place, but that seems | ||
| orthogonal to the design here. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this RFC under the umbrella of https://rust-lang.github.io/rust-project-goals/2026/in-place-init.html? Was this discussed on https://rust-lang.zulipchat.com/#narrow/channel/528918-t-lang.2Fin-place-init?
From my understanding, the Rust project is still in the evaluation stage for in-place initialization and is still working on their "design space RFC". I would recommend checking with the developers already in this space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The in-place-init project is aware of this. Having looked in detail at what in-place-init is discussing I would classify this as orthogonal to the concerns what they are discussing.
It does overlap somewhat with the custom-refs plans, however I believe this can be a good initial step that can provide results far faster than the complex field projection based designs they are producing.