Skip to content

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Dec 26, 2025

Summary by CodeRabbit

  • New Features

    • Added __itemsize__ property to type objects for introspection.
  • Bug Fixes & Improvements

    • Enhanced __slots__ validation with stricter error checking and conflict detection.
    • Improved type creation logic to prevent invalid slot redefinitions.
    • Built-in types (bytes, int, tuple) now properly declare memory layout metadata.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 26, 2025

📝 Walkthrough

Walkthrough

This PR introduces itemsize support to the RustPython type system. The itemsize attribute is added to the pyclass macro, propagated through the derive macro implementation to generate slots initialization code, and wired into the core type infrastructure. Built-in types (bytes, int, tuple) specify their itemsize, and a new __itemsize__ getter is added to PyType for introspection.

Changes

Cohort / File(s) Summary
Macro Implementation
crates/derive-impl/src/pyclass.rs
Extended __extend_slots generation to parse and apply optional itemsize attribute; inserts itemsize assignment before slots initialization in both impl and trait blocks via ExtractedImplAttrs.
Built-in Type Declarations
crates/vm/src/builtins/bytes.rs, crates/vm/src/builtins/int.rs, crates/vm/src/builtins/tuple.rs
Added itemsize to pyclass attributes: bytes (itemsize = 1), int (itemsize = 4), tuple (itemsize = sizeof PyObjectRef); reformatted tuple's with(...) list to single line.
Type System Infrastructure
crates/vm/src/types/slot.rs
Added new public field pub itemsize: usize to PyTypeSlots struct (replacing prior // tp_itemsize comment).
Class Definition Base
crates/vm/src/class.rs
Introduced const ITEMSIZE: usize = 0 in PyClassDef trait; propagated into PyTypeSlots initialization via make_slots().
Type Object Enhancement
crates/vm/src/builtins/type.rs
Added public __itemsize__() getter returning slots.itemsize; substantially expanded __slots__ processing with validation (duplicates, conflicts, base-type constraints), filters dict/weakref handling, and enhanced solid_base logic to detect extra instance variables via basicsize/itemsize comparison.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • RustPython#6515: Modifies derive macro's __extend_slots code generation in a related context (slot emission special-casing).

Suggested reviewers

  • arihant2math

Poem

🐰 Hop hop, itemsize takes flight!
Slots now sized just right,
From bytes to ints so neat,
Type introspection complete!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'tp_itemsize' directly references the core feature being implemented across multiple files, matching the primary objective of adding itemsize support to type slots.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

Code has been automatically formatted

The code in this PR has been formatted using cargo fmt --all.
Please pull the latest changes before pushing again:

git pull origin itemsize

@youknowone youknowone marked this pull request as ready for review December 27, 2025 16:07
@youknowone
Copy link
Member Author

@coderabbitai review

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 28, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
crates/vm/src/class.rs (1)

67-79: ITEMSIZE constant and slots.itemsize wiring look correct; clarify who owns it

Plumbing PyClassDef::ITEMSIZE into PyClassImpl::make_slots() is sane and keeps the default at 0 for existing types, while allowing manual impls to opt into a non‑zero tp_itemsize. At the same time, the derive macro now also sets slots.itemsize inside __extend_slots based on a #[pyclass(itemsize = ...)] on impl blocks.

That gives you two ways to set itemsize (the const and the impl attribute). Today the builtins (int/bytes/tuple) appear to rely only on the impl‑attribute path, leaving ITEMSIZE at its default. If that’s the intended split—ITEMSIZE for hand‑written classes, attribute for macro‑driven ones—it might be worth documenting to avoid future divergence between the two.

Also applies to: 209-225

crates/derive-impl/src/pyclass.rs (1)

162-207: itemsize pyimpl attribute is plumbed correctly into generated __extend_slots

The new flow—parsing itemsize = <expr> into ExtractedImplAttrs and emitting:

fn __extend_slots(slots: &mut ::rustpython_vm::types::PyTypeSlots) {
    slots.itemsize = <expr>;
    // existing slot extensions…
}

for the main impl works well for cases like #[pyclass(itemsize = 4)] impl PyInt { .. } and #[pyclass(itemsize = std::mem::size_of::<crate::PyObjectRef>())] impl PyTuple { .. }. Because this runs before trait extend_slots calls, trait-provided slots see the final itemsize.

Two minor edge cases worth noting:

  • itemsize used on a trait-level #[pyclass] impl is currently parsed but ignored, which is a bit non-obvious.
  • Multiple #[pyclass] impls for the same payload type could each specify itemsize, with the last one silently “winning”.

Neither is incorrect, but if you want clearer invariants you might later tighten this by rejecting itemsize on traits and/or detecting conflicting itemsize values across impls.

Also applies to: 1624-1650, 1735-1758

crates/vm/src/builtins/type.rs (2)

951-954: New __itemsize__ getter cleanly exposes the slot field

Exposing self.slots.itemsize via __itemsize__ is the right surface for Python-side introspection and lines up with how __basicsize__ is already handled. The implementation is trivial and side‑effect free; if you later want full CPython parity for subclasses, you may also want to consider whether heap types created via type() should inherit their base’s itemsize for introspection, not just for layout checks.


1355-1511: __slots__ / itemsize handling and solid_base layout checks look correct; a couple of edge cases to double‑check

The new type.__new__ path around __slots__ and itemsize is a substantial improvement:

  • Rejecting bytes as __slots__ or as individual slot items, and requiring identifiers via isidentifier(), matches Python’s expectations.
  • Treating any non‑"__dict__" / "__weakref__" entry as “custom” and forbidding such custom slots when base.slots.itemsize > 0 gives you the intended “nonempty slots not supported for subtype of ''” behavior for variable-sized bases like int, bytes, and tuple.
  • Tracking duplicates for __dict__ / __weakref__, preventing redefinition when the base already supplies them, and rejecting slot names that conflict with class attributes are all aligned with CPython’s rules.
  • Filtering "__dict__" out of the stored slot tuple while still setting HAS_DICT | MANAGED_DICT and then adding a __dict__ descriptor only when needed produces the right combination of layout and attribute behavior.
  • The updated solid_base using (basicsize, itemsize) to detect extra_ivars is more faithful to CPython’s layout checks and keeps best_base’s instance‑layout conflict detection consistent with the new itemsize plumbing.

Two subtleties worth verifying with tests:

  1. Weakref inheritance check – for built‑in bases, base_has_weakref is approximated as base.attributes.read().contains_key("__weakref__"). CPython looks up __weakref__ across the MRO, so a built‑in that only inherits the descriptor (rather than defining it in its own dict) might slip through here and allow an extra "__weakref__" slot. A quick CPython vs RustPython test around:

    class C(<some builtin with weakref support>):
        __slots__ = ("__weakref__",)

    would confirm whether behavior matches.

  2. Subclass __itemsize__ introspection – heap types created via type() currently start with itemsize == 0 even if their solid base has a non‑zero itemsize. That’s fine for layout/slots checks (you already consult the base’s itemsize), but their __itemsize__ values may not mirror CPython’s for subclasses of int/bytes/tuple if CPython exposes the base’s tp_itemsize there.

Functionally this all looks sound; the above points are mainly about nailing CPython compatibility in corner cases.

Also applies to: 1525-1564, 2175-2185

📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9b2ad34 and 8a89e3c.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_builtin.py is excluded by !Lib/**
📒 Files selected for processing (7)
  • crates/derive-impl/src/pyclass.rs
  • crates/vm/src/builtins/bytes.rs
  • crates/vm/src/builtins/int.rs
  • crates/vm/src/builtins/tuple.rs
  • crates/vm/src/builtins/type.rs
  • crates/vm/src/class.rs
  • crates/vm/src/types/slot.rs
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/vm/src/builtins/tuple.rs
  • crates/vm/src/builtins/int.rs
  • crates/vm/src/types/slot.rs
  • crates/vm/src/builtins/bytes.rs
  • crates/vm/src/class.rs
  • crates/derive-impl/src/pyclass.rs
  • crates/vm/src/builtins/type.rs
🧠 Learnings (4)
📓 Common learnings
Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.011Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
📚 Learning: 2025-08-26T05:20:54.540Z
Learnt from: youknowone
Repo: RustPython/RustPython PR: 6110
File: vm/src/frame.rs:1311-1316
Timestamp: 2025-08-26T05:20:54.540Z
Learning: In RustPython's pattern matching implementation, only certain builtin types should have the SEQUENCE flag: list and tuple are confirmed sequences. The user youknowone indicated that bytes, bytearray are not considered sequences in this context, even though they implement sequence-like protocols.

Applied to files:

  • crates/vm/src/builtins/int.rs
📚 Learning: 2025-08-26T05:20:54.540Z
Learnt from: youknowone
Repo: RustPython/RustPython PR: 6110
File: vm/src/frame.rs:1311-1316
Timestamp: 2025-08-26T05:20:54.540Z
Learning: In the RustPython codebase, only certain builtin types should be marked with the SEQUENCE flag for pattern matching. List and tuple are sequences, but bytes, bytearray, and range are not considered sequences in this context, even though they may implement sequence-like protocols.

Applied to files:

  • crates/vm/src/builtins/int.rs
📚 Learning: 2025-12-27T14:03:49.011Z
Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.011Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust

Applied to files:

  • crates/derive-impl/src/pyclass.rs
🧬 Code graph analysis (1)
crates/vm/src/builtins/bytes.rs (2)
crates/vm/src/builtins/memory.rs (1)
  • itemsize (584-586)
crates/stdlib/src/array.rs (1)
  • itemsize (738-740)
🔇 Additional comments (4)
crates/vm/src/builtins/int.rs (1)

322-326: itemsize = 4 for int looks correct but ties you to BigInt’s current layout

Using itemsize = 4 makes int a variable-sized type for layout checks and __itemsize__ introspection, matching a u32‑digit representation. If malachite’s internal digit size ever changes, this constant will need to be revisited (and tests around int.__itemsize__/type(int).slots.itemsize would catch that).

crates/vm/src/builtins/bytes.rs (1)

189-205: itemsize = 1 on bytes is consistent with element size and flag usage

Marking bytes with itemsize = 1 aligns tp_itemsize with the per‑byte element size and lets the type/slots machinery treat it as variable‑sized, without changing SEQUENCE flags (bytes still only has BASETYPE and _MATCH_SELF, which is consistent with earlier decisions about sequence flags).

crates/vm/src/types/slot.rs (1)

120-187: New itemsize field cleanly exposes tp_itemsize in PyTypeSlots

Adding pub itemsize: usize with a Default of 0 and wiring it through PyTypeSlots::new/heap_default is sound; types that care about variable-size layout now have an explicit slot to populate, while existing code that doesn’t touch itemsize continues to see a fixed-size (0) default.

crates/vm/src/builtins/tuple.rs (1)

259-263: Tuple itemsize correctly reflects per-element pointer size

Setting itemsize = std::mem::size_of::<crate::PyObjectRef>() makes tuple.__itemsize__ line up with the pointer-sized element layout and lets the type system treat tuples as variable-sized for layout/slots checks. Flags still include SEQUENCE, which matches the earlier guidance that only list and tuple should carry the SEQUENCE flag for pattern matching.

@youknowone youknowone merged commit 7edc3bb into RustPython:main Dec 28, 2025
13 checks passed
@youknowone youknowone deleted the itemsize branch December 28, 2025 01:27
@coderabbitai coderabbitai bot mentioned this pull request Dec 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant