Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@
^\.jules(/.*)?$
^\.trivyignore\.yaml$
^trivy\.yaml$
^\.semgrepignore$
3 changes: 3 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,6 @@
## 2025-02-12 - R 언어에서 반복적인 mirt 모델 생성 시 불필요한 데이터프레임 부분집합 추출 최적화
**Learning:** R에서 데이터프레임의 특정 열을 추출하는 작업(`df[cols]`)은 O(N)의 메모리 복사를 수반합니다. `autoFIPC`에서 `mirt` 모델의 파라미터를 설정하거나 호출하는 과정 중에 `newformXDataK[colnames(newFormModel@Data$data)]` 코드가 반복해서 사용되었고, 심지어 `ncol()`을 위해 단순히 개수를 구할 때도 사용되어 불필요한 메모리 할당과 오버헤드를 초래했습니다.
**Action:** 조건문이나 반복문 내부에서 불필요하게 데이터프레임 부분집합 연산이 반복되지 않도록 외부에서 한 번만 `linkedFormData <- newformXDataK[colnames(newFormModel@Data$data)]`로 캐싱(caching)한 뒤, `ncol(linkedFormData)`와 `data = linkedFormData` 형태로 재사용하여 메모리 복사와 O(N) 오버헤드를 방지해야 합니다.
## 2024-07-23 - R 언어에서 열 이름 추출 시 데이터프레임 부분집합 추출을 피하여 O(N) 메모리 복사 방지
**Learning:** R에서 열 이름을 확인하기 위해 `colnames(df[cols])` 형태로 데이터프레임을 서브셋팅하면, 단순히 이름만 추출하는 경우에도 데이터를 복사하는 과정에서 불필요한 O(N) 메모리 할당과 복사 오버헤드가 발생합니다.
**Action:** 열 이름을 추출하거나 비교할 때는 서브셋팅 대신 `intersect(cols, colnames(df))` 함수를 사용하여 데이터 복사 없이 O(1) 수준으로 성능을 개선해야 합니다.
Comment on lines +19 to +21
10 changes: 5 additions & 5 deletions R/aFIPC.R
Original file line number Diff line number Diff line change
Expand Up @@ -620,8 +620,8 @@ autoFIPC <-
IPDItemCount <- 0

# IPD target item checking
newFormColNames <- colnames(newformXDataK[colnames(newFormModel@Data$data)])
oldFormColNames <- colnames(oldformYDataK[colnames(oldFormModel@Data$data)])
newFormColNames <- intersect(colnames(newFormModel@Data$data), colnames(newformXDataK))
oldFormColNames <- intersect(colnames(oldFormModel@Data$data), colnames(oldformYDataK))
Comment on lines +623 to +624

# ⚡ Bolt: Vectorized match() to avoid dynamic array growth overhead inside a for loop
idxNew <- match(newformCommonItemNames, newFormColNames)
Expand Down Expand Up @@ -749,8 +749,8 @@ autoFIPC <-
}
}

newFormColNames <- colnames(newformXDataK[colnames(newFormModel@Data$data)])
oldFormColNames <- colnames(oldformYDataK[colnames(oldFormModel@Data$data)])
newFormColNames <- intersect(colnames(newFormModel@Data$data), colnames(newformXDataK))
oldFormColNames <- intersect(colnames(oldFormModel@Data$data), colnames(oldformYDataK))

# ⚡ Bolt: Cache parameter indices to avoid O(N) linear search inside loop
newScaleParmsItemIdxCache <- split(seq_len(nrow(NewScaleParms)), NewScaleParms$item)
Expand Down Expand Up @@ -848,7 +848,7 @@ autoFIPC <-
message('\nestimating Linked Form Eq(X) parameters')

# ⚡ Bolt: Cache subsetted dataframe to avoid repeated O(N) memory copies during mirt model setup
linkedFormData <- newformXDataK[colnames(newFormModel@Data$data)]
linkedFormData <- newformXDataK[, intersect(colnames(newFormModel@Data$data), colnames(newformXDataK)), drop = FALSE]

if (forceNormalZeroOne) {
freeMEAN <- F
Expand Down
2 changes: 0 additions & 2 deletions test_dummy.R

This file was deleted.

3 changes: 0 additions & 3 deletions test_validation.R

This file was deleted.

30 changes: 30 additions & 0 deletions tests/testthat/test-bolt-intersect.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
test_that("intersect column name extraction works properly in autoFIPC", {
skip_if_not_installed("mirt")
set.seed(42)

a <- matrix(c(1, 1.2, 0.8, 1.5, 0.9, 1.1, 1.0, 1.3), ncol=1)
d <- matrix(c(1, -1, 0, 0.5, -0.5, 0, 0.2, -0.2), ncol=1)
oldformYData <- mirt::simdata(a, d, 250, itemtype = '2PL')
colnames(oldformYData) <- paste0("Item", 1:8)

a2 <- matrix(c(1, 1.2, 0.8, 1.0, 1.3), ncol=1)
d2 <- matrix(c(1, -1, 0, 0.2, -0.2), ncol=1)
newformXData <- mirt::simdata(a2, d2, 250, itemtype = '2PL')
colnames(newformXData) <- c("Item1", "Item2", "Item3", "NewItem1", "NewItem2")

result <- aFIPC::autoFIPC(
newformXData = newformXData,
oldformYData = oldformYData,
newformCommonItemNames = c('Item1', 'Item2', 'Item3'),
oldformCommonItemNames = c('Item1', 'Item2', 'Item3'),
confirmCommonItems = FALSE, # This skips interactive confirmation
tryEM = TRUE,
freeMEAN = FALSE,
Comment on lines +18 to +22
forceNormalZeroOne = FALSE,
tryFitwholeOldItems = FALSE,
tryFitwholeNewItems = FALSE
)

expect_true(!is.null(result$LinkedModel))
expect_true(methods::is(result$LinkedModel, "SingleGroupClass"))
})
Loading