-
Notifications
You must be signed in to change notification settings - Fork 1.4k
tp_itemsize #6544
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tp_itemsize #6544
Conversation
📝 WalkthroughWalkthroughThis PR introduces Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Code has been automatically formatted The code in this PR has been formatted using git pull origin itemsize |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (4)
crates/vm/src/class.rs (1)
67-79: ITEMSIZE constant andslots.itemsizewiring look correct; clarify who owns itPlumbing
PyClassDef::ITEMSIZEintoPyClassImpl::make_slots()is sane and keeps the default at 0 for existing types, while allowing manual impls to opt into a non‑zerotp_itemsize. At the same time, the derive macro now also setsslots.itemsizeinside__extend_slotsbased on a#[pyclass(itemsize = ...)]on impl blocks.That gives you two ways to set
itemsize(the const and the impl attribute). Today the builtins (int/bytes/tuple) appear to rely only on the impl‑attribute path, leaving ITEMSIZE at its default. If that’s the intended split—ITEMSIZE for hand‑written classes, attribute for macro‑driven ones—it might be worth documenting to avoid future divergence between the two.Also applies to: 209-225
crates/derive-impl/src/pyclass.rs (1)
162-207:itemsizepyimpl attribute is plumbed correctly into generated__extend_slotsThe new flow—parsing
itemsize = <expr>intoExtractedImplAttrsand emitting:fn __extend_slots(slots: &mut ::rustpython_vm::types::PyTypeSlots) { slots.itemsize = <expr>; // existing slot extensions… }for the main
implworks well for cases like#[pyclass(itemsize = 4)] impl PyInt { .. }and#[pyclass(itemsize = std::mem::size_of::<crate::PyObjectRef>())] impl PyTuple { .. }. Because this runs before traitextend_slotscalls, trait-provided slots see the finalitemsize.Two minor edge cases worth noting:
itemsizeused on a trait-level #[pyclass] impl is currently parsed but ignored, which is a bit non-obvious.- Multiple #[pyclass] impls for the same payload type could each specify
itemsize, with the last one silently “winning”.Neither is incorrect, but if you want clearer invariants you might later tighten this by rejecting
itemsizeon traits and/or detecting conflictingitemsizevalues across impls.Also applies to: 1624-1650, 1735-1758
crates/vm/src/builtins/type.rs (2)
951-954: New__itemsize__getter cleanly exposes the slot fieldExposing
self.slots.itemsizevia__itemsize__is the right surface for Python-side introspection and lines up with how__basicsize__is already handled. The implementation is trivial and side‑effect free; if you later want full CPython parity for subclasses, you may also want to consider whether heap types created viatype()should inherit their base’s itemsize for introspection, not just for layout checks.
1355-1511:__slots__/itemsizehandling andsolid_baselayout checks look correct; a couple of edge cases to double‑checkThe new
type.__new__path around__slots__anditemsizeis a substantial improvement:
- Rejecting bytes as
__slots__or as individual slot items, and requiring identifiers viaisidentifier(), matches Python’s expectations.- Treating any non‑
"__dict__"/"__weakref__"entry as “custom” and forbidding such custom slots whenbase.slots.itemsize > 0gives you the intended “nonempty slots not supported for subtype of ''” behavior for variable-sized bases likeint,bytes, andtuple.- Tracking duplicates for
__dict__/__weakref__, preventing redefinition when the base already supplies them, and rejecting slot names that conflict with class attributes are all aligned with CPython’s rules.- Filtering
"__dict__"out of the stored slot tuple while still settingHAS_DICT | MANAGED_DICTand then adding a__dict__descriptor only when needed produces the right combination of layout and attribute behavior.- The updated
solid_baseusing(basicsize, itemsize)to detectextra_ivarsis more faithful to CPython’s layout checks and keepsbest_base’s instance‑layout conflict detection consistent with the new itemsize plumbing.Two subtleties worth verifying with tests:
Weakref inheritance check – for built‑in bases,
base_has_weakrefis approximated asbase.attributes.read().contains_key("__weakref__"). CPython looks up__weakref__across the MRO, so a built‑in that only inherits the descriptor (rather than defining it in its own dict) might slip through here and allow an extra"__weakref__"slot. A quick CPython vs RustPython test around:class C(<some builtin with weakref support>): __slots__ = ("__weakref__",)would confirm whether behavior matches.
Subclass
__itemsize__introspection – heap types created viatype()currently start withitemsize == 0even if their solid base has a non‑zero itemsize. That’s fine for layout/slots checks (you already consult the base’s itemsize), but their__itemsize__values may not mirror CPython’s for subclasses ofint/bytes/tupleif CPython exposes the base’stp_itemsizethere.Functionally this all looks sound; the above points are mainly about nailing CPython compatibility in corner cases.
Also applies to: 1525-1564, 2175-2185
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
Lib/test/test_builtin.pyis excluded by!Lib/**
📒 Files selected for processing (7)
crates/derive-impl/src/pyclass.rscrates/vm/src/builtins/bytes.rscrates/vm/src/builtins/int.rscrates/vm/src/builtins/tuple.rscrates/vm/src/builtins/type.rscrates/vm/src/class.rscrates/vm/src/types/slot.rs
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Follow the default rustfmt code style by runningcargo fmtto format Rust code
Always run clippy to lint code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass,pymodule,pyfunction, etc.) when implementing Python functionality in Rust
Files:
crates/vm/src/builtins/tuple.rscrates/vm/src/builtins/int.rscrates/vm/src/types/slot.rscrates/vm/src/builtins/bytes.rscrates/vm/src/class.rscrates/derive-impl/src/pyclass.rscrates/vm/src/builtins/type.rs
🧠 Learnings (4)
📓 Common learnings
Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.011Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
📚 Learning: 2025-08-26T05:20:54.540Z
Learnt from: youknowone
Repo: RustPython/RustPython PR: 6110
File: vm/src/frame.rs:1311-1316
Timestamp: 2025-08-26T05:20:54.540Z
Learning: In RustPython's pattern matching implementation, only certain builtin types should have the SEQUENCE flag: list and tuple are confirmed sequences. The user youknowone indicated that bytes, bytearray are not considered sequences in this context, even though they implement sequence-like protocols.
Applied to files:
crates/vm/src/builtins/int.rs
📚 Learning: 2025-08-26T05:20:54.540Z
Learnt from: youknowone
Repo: RustPython/RustPython PR: 6110
File: vm/src/frame.rs:1311-1316
Timestamp: 2025-08-26T05:20:54.540Z
Learning: In the RustPython codebase, only certain builtin types should be marked with the SEQUENCE flag for pattern matching. List and tuple are sequences, but bytes, bytearray, and range are not considered sequences in this context, even though they may implement sequence-like protocols.
Applied to files:
crates/vm/src/builtins/int.rs
📚 Learning: 2025-12-27T14:03:49.011Z
Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.011Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
Applied to files:
crates/derive-impl/src/pyclass.rs
🧬 Code graph analysis (1)
crates/vm/src/builtins/bytes.rs (2)
crates/vm/src/builtins/memory.rs (1)
itemsize(584-586)crates/stdlib/src/array.rs (1)
itemsize(738-740)
🔇 Additional comments (4)
crates/vm/src/builtins/int.rs (1)
322-326:itemsize = 4forintlooks correct but ties you to BigInt’s current layoutUsing
itemsize = 4makesinta variable-sized type for layout checks and__itemsize__introspection, matching a u32‑digit representation. If malachite’s internal digit size ever changes, this constant will need to be revisited (and tests aroundint.__itemsize__/type(int).slots.itemsizewould catch that).crates/vm/src/builtins/bytes.rs (1)
189-205:itemsize = 1onbytesis consistent with element size and flag usageMarking
byteswithitemsize = 1aligns tp_itemsize with the per‑byte element size and lets the type/slots machinery treat it as variable‑sized, without changing SEQUENCE flags (bytes still only has BASETYPE and_MATCH_SELF, which is consistent with earlier decisions about sequence flags).crates/vm/src/types/slot.rs (1)
120-187: Newitemsizefield cleanly exposestp_itemsizeinPyTypeSlotsAdding
pub itemsize: usizewith aDefaultof 0 and wiring it throughPyTypeSlots::new/heap_defaultis sound; types that care about variable-size layout now have an explicit slot to populate, while existing code that doesn’t touchitemsizecontinues to see a fixed-size (0) default.crates/vm/src/builtins/tuple.rs (1)
259-263: Tupleitemsizecorrectly reflects per-element pointer sizeSetting
itemsize = std::mem::size_of::<crate::PyObjectRef>()makestuple.__itemsize__line up with the pointer-sized element layout and lets the type system treat tuples as variable-sized for layout/slots checks. Flags still includeSEQUENCE, which matches the earlier guidance that only list and tuple should carry the SEQUENCE flag for pattern matching.
Summary by CodeRabbit
New Features
__itemsize__property to type objects for introspection.Bug Fixes & Improvements
__slots__validation with stricter error checking and conflict detection.✏️ Tip: You can customize this high-level summary in your review settings.