Skip to content

move the cedta fallback outside of [ #7936

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ben-schwen wants to merge 1 commit into from
Open

move the cedta fallback outside of [ #7936

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 24 additions & 16 deletions R/data.table.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -521,11 +521,31 @@ replace_dot_alias = function(e) {
list(GForce=GForce, jsub=jsub, jvnames=jvnames)
}

"[.data.table" = function(x, i, j, by, keyby, with=TRUE, nomatch=NA, mult="all", roll=FALSE, rollends=if (roll=="nearest") c(TRUE,TRUE) else if (roll>=0.0) c(FALSE,TRUE) else c(TRUE,FALSE), which=FALSE, .SDcols, verbose=getOption("datatable.verbose"), allow.cartesian=getOption("datatable.allow.cartesian"), drop=NULL, on=NULL, env=NULL, showProgress=getOption("datatable.showProgress", interactive()))
# [ called from a scope that is not data.table-aware (!cedta()) behaves like [.data.frame.
# i, j and drop are passed on as promises, so their missingness reaches [.data.frame unchanged.
# nargs is nargs() of the [.data.table call.
.cedta_fallback = function(x, i, j, drop, jsub, nargs) {
# Fix for #500 (to do)
if (jsub %iscall% c(":=", "let")) {
# Throw a specific error message
stopf("[ was called on a data.table in an environment that is not data.table-aware (i.e. cedta()), but '%s' was used, implying the owner of this call really intended for data.table methods to be called. See vignette('datatable-importing') for details on properly importing data.table.", as.character(jsub[[1L]]))
}
missing_drop = missing(drop)
Nargs = nargs - (!missing_drop)

@MichaelChirico MichaelChirico Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we re-compute missing(drop) here? (and missing_i?)

Since this is an "unsupported" path it's fine to have a little overhead, I think it's preferable to have a minimal helper signature.

I also wonder if a do.call(`[.data.frame`, list(...)) approach could simplify(?) this arg-checking logic.

@ben-schwen ben-schwen Nov 3, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We can recompute by also dropping drop=NULL default from [.

do.call(`[.data.frame`, list(...)) would still need to force i on list(DT, i, j) and would therefore error

ans = if (Nargs<3L) { `[.data.frame`(x,i) } # drop ignored anyway by DF[i]
else if (missing_drop) `[.data.frame`(x,i,j)
else `[.data.frame`(x,i,j,drop)
# added is.data.table(ans) check to fix bug #81
if (!missing(i) && is.data.table(ans)) setkey(ans, NULL) # drops index too; tested by plyr::arrange test in other.Rraw
ans
}
class(.cedta_fallback) = "dt_query_helper"

"[.data.table" = function(x, i, j, by, keyby, with=TRUE, nomatch=NA, mult="all", roll=FALSE, rollends=if (roll=="nearest") c(TRUE,TRUE) else if (roll>=0.0) c(FALSE,TRUE) else c(TRUE,FALSE), which=FALSE, .SDcols, verbose=getOption("datatable.verbose"), allow.cartesian=getOption("datatable.allow.cartesian"), drop, on=NULL, env=NULL, showProgress=getOption("datatable.showProgress", interactive()))
{
# ..selfcount <<- ..selfcount+1 # in dev, we check no self calls, each of which doubles overhead, or could
# test explicitly if the caller is [.data.table (even stronger test. TO DO.)
# the drop=NULL is to sink drop argument when dispatching to [.data.frame; using '...' stops test 147
# the drop is to sink drop argument when dispatching to [.data.frame; using '...' stops test 147
# missing() is only valid in this frame, so it is captured once here and code moved into helpers uses these flags, #852.
# Keep them in sync when an argument is reassigned below: assigning a value clears missingness, `x = substitute()` restores it.
missing_i = missing(i)
Expand All @@ -539,20 +559,8 @@ replace_dot_alias = function(e) {
missing_verbose = missing(verbose)
# capture parent.frame() changes once as calling scope
callenv = parent.frame()
if (!cedta()) {
# Fix for #500 (to do)
if (substitute(j) %iscall% c(":=", "let")) {
# Throw a specific error message
stopf("[ was called on a data.table in an environment that is not data.table-aware (i.e. cedta()), but '%s' was used, implying the owner of this call really intended for data.table methods to be called. See vignette('datatable-importing') for details on properly importing data.table.", as.character(substitute(j)[[1L]]))
}
Nargs = nargs() - (!missing_drop)
ans = if (Nargs<3L) { `[.data.frame`(x,i) } # drop ignored anyway by DF[i]
else if (missing_drop) `[.data.frame`(x,i,j)
else `[.data.frame`(x,i,j,drop)
# added is.data.table(ans) check to fix bug #81
if (!missing_i && is.data.table(ans)) setkey(ans, NULL) # drops index too; tested by plyr::arrange test in other.Rraw
return(ans)
}
# cedta() and nargs() depend on this frame so are evaluated here
if (!cedta()) return(.cedta_fallback(x, i, j, drop, substitute(j), nargs()))
if (!missing_verbose) {
if (!is.integer(verbose) && !is.logical(verbose)) stopf("verbose must be logical or integer")
if (length(verbose)!=1L || anyNA(verbose)) stopf("verbose must be length 1 non-NA")
Expand Down
18 changes: 18 additions & 0 deletions inst/tests/bracket.Rraw
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,3 +171,21 @@ test(2.48, X[, lapply(.SD, sum), .SDcols=is.integer], data.table(id=7L, v=10L))
test(2.49, X[, lapply(.SD, sum), .SDcols=!"id"], data.table(t=12, v=10L))
test(2.50, X[, lapply(.SD, sum), .SDcols=-(id:t)], data.table(v=10L))
test(2.51, X[, lapply(.SD, function(x) x/v), .SDcols=!"v"], data.table(id=c(1,0.5,2/3,0.75), t=c(1,2.5,2/3,1)))

# [ from a scope that is not data.table-aware dispatches to [.data.frame, #852
DT = data.table(a=1:3, b=c("x","y","z"), key="b")
DF = as.data.frame(DT)
naive = function(expr) eval(substitute(expr), list(DT=DT), new.env(parent=asNamespace("base"))) # use base as namespace to be sure its installed
# results of [.data.frame are not over-allocated, hence compared as data.frame
test(3.01, setDF(naive(DT[2])), DF[2])
test(3.02, setDF(naive(DT[1:2,])), DF[1:2,]) # slice from front for rownames
test(3.03, setDF(naive(DT[, 2, drop=FALSE])), DF[, 2, drop=FALSE])
test(3.04, naive(DT[, 2]), DF[, 2])
test(3.05, naive(DT[2, "a"]), DF[2, "a"])
test(3.06, key(naive(DT[])), "b") # key retained when i is missing, dropped otherwise
test(3.07, key(naive(DT[1:3, ])), NULL)
test(3.08, naive(DT[, a := 1L]), error="[ was called on a data.table in an environment that is not data.table-aware (i.e. cedta()), but ':=' was used")
test(3.09, naive(DT[, let(a = 1L)]), error="but 'let' was used")
test(3.10, class(naive(DT[2:3, ])), c("data.table","data.frame"))
test(3.11, tryCatch(naive(DT[, a := 1L]), error=conditionCall)[[1L]], quote(`[.data.table`)) # not .cedta_fallback
rm(DT, DF, naive)
2 changes: 1 addition & 1 deletion man/data.table.Rd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ data.table(\dots, keep.rownames=FALSE, check.names=FALSE, key=NULL, stringsAsFac
.SDcols,
verbose = getOption("datatable.verbose"), # default: FALSE
allow.cartesian = getOption("datatable.allow.cartesian"), # default: FALSE
drop = NULL, on = NULL, env = NULL,
drop, on = NULL, env = NULL,
showProgress = getOption("datatable.showProgress", interactive()))
}
\arguments{
Expand Down
Loading