feat: add `ml/strided/dsgd-trainer` by nakul-krishnakumar · Pull Request #13614 · stdlib-js/stdlib

nakul-krishnakumar · 2026-07-22T20:22:37Z

Resolves a part of #12875.

Description

What is the purpose of this pull request?

This pull request:

Adds ml/strided/dsgd-trainer.

Related Issues

Does this pull request have any related issues?

This pull request has the following related issues:

[RFC]: Add batch machine learning algorithms in Javascript and C (tracking issue) #12875.

Questions

Any questions for reviewers of this pull request?

No.

Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

No.

Checklist

Please ensure the following tasks are completed before submitting this pull request.

Read, understood, and followed the contributing guidelines.

AI Assistance

When authoring the changes proposed in this PR, did you use any kind of AI assistance?

Yes
No

If you answered "yes" above, how did you use AI assistance?

Code generation (e.g., when writing an implementation or fixing a bug)
Test/benchmark generation
Documentation (including examples)
Research and understanding

Disclosure

If you answered "yes" to using AI assistance, please provide a short disclosure indicating how you used AI assistance. This helps reviewers determine how much scrutiny to apply when reviewing your contribution. Example disclosures: "This PR was written primarily by Claude Code." or "I consulted ChatGPT to understand the codebase, but the proposed changes were fully authored manually by myself.".

I used both Claude Code and ChatGPT to help me refer the sklearn implementation and write the code.

@stdlib-js/reviewers

nakul-krishnakumar · 2026-07-22T20:36:17Z

I have the following doubts:

As we are anyway having branching for penalties and learningRate, I think it would be okay to keep a single API ml/strided/sgd-trainer with branching for lossFunctions as well as instead of having separate sgd trainers per loss function as we had planned on in the earlier meetings (dsgd-trainer-squared-epsilon-insensitive).
Should we implement the helper function l1Penalty as a standalone implementation? ( I don't see any reason why we shouldn't )
Should the API support all 3 penalties/regularization? (l1, l2 and elasticnet)
- ml/incr/binary-classification supports only L2 by default.
- If going for all 3, we will have to implement standalone ml/base/sgd-classification/penalties (enum) (opened a PR feat: add ml/base/sgd-classification/penalties #13623 )
- I would suggest to have support for all 3.
How do we handle returning intercept? We cant just update the user passed intercept as we do for the weight matrix as I believe numbers are not passed by reference unlike arrays.
Currently this implementation is not in 1:1 parity with the sklearn API, it skips things like dataset shuffling, learning rate schedulers like PA1, PA2 (they were recently added in sklearn API, so I am skeptical on adding those), early stopping etc. It would be helpful if you could let me know if I am missing anything important here that you think would be helpful to have.
Should we have a weight vector implementation similar to how sklearn as well as ml/incr/sgd-regression does?
sklearn using weight vector in sgd implementation: Ref
Tidy up function argument, what all should be passed to base implementation?

sklearn sgd implementation: Ref

cc: @kgryte

nakul-krishnakumar · 2026-07-22T20:37:26Z

Also please note that I kept some comments only for the sake of understanding, will remove them once we finalize the implementation.

kgryte · 2026-07-22T23:17:07Z

Quick aside: I am wondering if we should namespace the strided routines. E.g.,

ml/strided/kmeans/ddistance
ml/strided/sgd-trainer/dsquared-epsilon-insensitive
...

The current package base name, as is, is a bit unwieldy. Thoughts?

kgryte · 2026-07-22T23:19:35Z

For packages which are applicable across ML algos, they can go directly in ml/strided. E.g., hypothetically,

ml/strided/dsquared-epsilon-insensitive

And actually, is squared epsilon insensitive so specific to sgd-trainer that it shouldn't be broadly applicable?

I am just curious where we draw the line.

nakul-krishnakumar · 2026-07-23T04:46:24Z

Quick aside: I am wondering if we should namespace the strided routines. E.g.,
ml/strided/kmeans/ddistance
ml/strided/sgd-trainer/dsquared-epsilon-insensitive
...
The current package base name, as is, is a bit unwieldy. Thoughts?

Yeah we could do that, we had discussed this earlier but decided not to until we add more packages and it becomes harder to track.

nakul-krishnakumar · 2026-07-23T04:53:06Z

For packages which are applicable across ML algos, they can go directly in ml/strided. E.g., hypothetically,
ml/strided/dsquared-epsilon-insensitive
And actually, is squared epsilon insensitive so specific to sgd-trainer that it shouldn't be broadly applicable?

I am just curious where we draw the line.

Here, yes it is specific to SGD. Because ml/strided/dsquared-epsilon-insensitive is not actually a strided version of squared-epsilon-insensitive loss function gradient, instead it is a strided version of SGD implementation hard coded with squared-epsilon-insensitive loss function.
Instead what I am saying is lets just have a single ml/strided/dsgd-trainer which uses a package similar to ml/strided/dkmeans-distance and resolve the gradient of the user passed loss function.

function dsgdTrainer( loss, .... ) {
    ...
    g = dsgdLossGradient( loss, x, e, y, p );
    ...
}

This was our first choice but decided on implementing individual sgd-trainer per loss function so that we can optimize it (as this doesn't require resolving the loss function using branching). But now that I look at it, the implementation anyway does branching to resolve learning rate and penalty so maybe branching for resolving loss functions too might not be that costly comparatively.

cc: @kgryte

nakul-krishnakumar · 2026-08-05T13:36:26Z

+*
+* Note:
+*
+* -   Here `params` => `[ ]` (empty)


Suggested change

* - Here `params` => `[ ]` (empty)

* - Here `params` => `[ eta0 ]` (empty)

nakul-krishnakumar · 2026-08-05T13:36:47Z

+* -   Here `params` => `[ lambda ]`
+*
+* @private
+* @param {NonNegativeInteger} t - current iteration.
+* @param {Float64Array} params - strided array containing scheduler specific parameters.
+* @returns {number} learning rate
+*/
+function pegasos( t, params ) {
+	return 1.0 / ( params[ 0 ]*t );
+}


Suggested change

* - Here `params` => `[ lambda ]`

*

* @private

* @param {NonNegativeInteger} t - current iteration.

* @param {Float64Array} params - strided array containing scheduler specific parameters.

* @returns {number} learning rate

*/

function pegasos( t, params ) {

return 1.0 / ( params[ 0 ]*t );

}

* - Here `params` => `[ eta0, lambda ]`

*

* @private

* @param {NonNegativeInteger} t - current iteration.

* @param {Float64Array} params - strided array containing scheduler specific parameters.

* @returns {number} learning rate

*/

function pegasos( t, params ) {

return 1.0 / ( params[ 1 ]*t );

}

nakul-krishnakumar · 2026-08-05T13:37:20Z

+* -   Here `params` => `[ eta, lambda, l1Ratio ]`
+*
+* @private
+* @param {NonNegativeInteger} scaleFactor - current iteration.
+* @param {Float64Array} params - strided array containing regularizer specific parameters.
+* @returns {number} scale factor
+*/
+function l2Decay( scaleFactor, params ) {
+	return scaleFactor * max( 0.0, 1.0 - ( ( 1.0 - params[ 2 ] ) * params[ 0 ] * params[ 1 ] ) ); // eslint-disable-line max-len
+}


Suggested change

* - Here `params` => `[ eta, lambda, l1Ratio ]`

*

* @private

* @param {NonNegativeInteger} scaleFactor - current iteration.

* @param {Float64Array} params - strided array containing regularizer specific parameters.

* @returns {number} scale factor

*/

function l2Decay( scaleFactor, params ) {

return scaleFactor * max( 0.0, 1.0 - ( ( 1.0 - params[ 2 ] ) * params[ 0 ] * params[ 1 ] ) ); // eslint-disable-line max-len

}

* - Here `params` => `[ lambda, l1Ratio ]`

*

* @private

* @param {NonNegativeInteger} scaleFactor - current iteration.

* @param {Float64Array} params - strided array containing regularizer specific parameters.

* @returns {number} scale factor

*/

function l2Decay( scaleFactor, eta, params ) {

return scaleFactor * max( 0.0, 1.0 - ( ( 1.0 - params[ 2 ] ) * eta * params[ 1 ] ) ); // eslint-disable-line max-len

}

nakul-krishnakumar · 2026-08-05T13:38:12Z

+function identityDecay( scaleFactor, params ) { // eslint-disable-line no-unused-vars
+	return scaleFactor;
+}


Suggested change

function identityDecay( scaleFactor, params ) { // eslint-disable-line no-unused-vars

return scaleFactor;

}

function identityDecay( scaleFactor, eta, params ) { // eslint-disable-line no-unused-vars

return scaleFactor;

}

nakul-krishnakumar · 2026-08-05T13:54:57Z

+	if ( penalty === 'l2' ) {
+		l1Ratio = 0.0;
+	} else if ( penalty === 'l1' ) {
+		l1Ratio = 1.0;
+	}


Suggested change

if ( penalty === 'l2' ) {

l1Ratio = 0.0;

} else if ( penalty === 'l1' ) {

l1Ratio = 1.0;

}

if ( penalty === 'l2' ) {

penaltyParams[ 1 ] = 0.0;

} else if ( penalty === 'l1' ) {

penaltyParams[ 1 ] = 1.0;

}

--- type: pre_commit_static_analysis_report description: Results of running static analysis checks when committing changes. report: - task: lint_filenames status: passed - task: lint_editorconfig status: passed - task: lint_markdown_pkg_readmes status: na - task: lint_markdown_docs status: na - task: lint_markdown status: na - task: lint_package_json status: na - task: lint_repl_help status: na - task: lint_javascript_src status: na - task: lint_javascript_cli status: na - task: lint_javascript_examples status: na - task: lint_javascript_tests status: na - task: lint_javascript_benchmarks status: na - task: lint_python status: na - task: lint_r status: na - task: lint_c_src status: na - task: lint_c_examples status: na - task: lint_c_benchmarks status: na - task: lint_c_tests_fixtures status: na - task: lint_shell status: na - task: lint_typescript_declarations status: passed - task: lint_typescript_tests status: na - task: lint_license_headers status: passed ---

kgryte · 2026-08-06T18:35:55Z

+/**
+* Struct for storing SGD parameters.
+*/
+struct stdlib_ml_sgd_params_float64_params {


Suggested change

struct stdlib_ml_sgd_params_float64_params {

struct stdlib_ml_sgd_float64_params {

feat: add ml/strided/dsgd-trainer-squared-epsilon-insensitive

db1a5fd

nakul-krishnakumar requested a review from a team July 22, 2026 20:22

nakul-krishnakumar marked this pull request as draft July 22, 2026 20:22

stdlib-bot added Needs Review A pull request which needs code review. and removed Needs Review A pull request which needs code review. labels Jul 22, 2026

nakul-krishnakumar added Machine Learning Issue or pull request specific to machine learning functionality. GSoC Google Summer of Code. gsoc: 2026 Google Summer of Code (2026). labels Jul 22, 2026

nakul-krishnakumar changed the title ~~feat: add ml/strided/dsgd-trainer-squared-epsilon-insensitive~~ feat: add ml/strided/dsgd-trainer Jul 24, 2026

nakul-krishnakumar mentioned this pull request Jul 27, 2026

Weekly Standup (2026-07-27) stdlib-js/meetings#145

Open

fix: use enum resolution objects

19e0707

nakul-krishnakumar commented Aug 5, 2026

View reviewed changes

nakul-krishnakumar added 3 commits August 6, 2026 12:38

Merge branch 'develop' into ml-sgd-sq-eps

0620f04

feat: add benchmarks, docs, C header

6e95f99

kgryte reviewed Aug 6, 2026

View reviewed changes

Merge branch 'develop' into ml-sgd-sq-eps

0ec508a

Navigation Menu

Uh oh!

Uh oh!

feat: add `ml/strided/dsgd-trainer` - #13614

feat: add `ml/strided/dsgd-trainer`#13614
nakul-krishnakumar wants to merge 6 commits into
stdlib-js:developfrom
nakul-krishnakumar:ml-sgd-sq-eps

nakul-krishnakumar commented Jul 22, 2026 •

edited

Loading

Uh oh!

nakul-krishnakumar commented Jul 22, 2026 •

edited

Loading

Uh oh!

nakul-krishnakumar commented Jul 22, 2026 •

edited

Loading

Uh oh!

kgryte commented Jul 22, 2026

Uh oh!

kgryte commented Jul 22, 2026

Uh oh!

nakul-krishnakumar commented Jul 23, 2026

Uh oh!

nakul-krishnakumar commented Jul 23, 2026 •

edited

Loading

Uh oh!

nakul-krishnakumar Aug 5, 2026

Uh oh!

nakul-krishnakumar Aug 5, 2026

Uh oh!

nakul-krishnakumar Aug 5, 2026

Uh oh!

nakul-krishnakumar Aug 5, 2026

Uh oh!

nakul-krishnakumar Aug 5, 2026

Uh oh!

kgryte Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	* - Here `params` => `[ ]` (empty)
	* - Here `params` => `[ eta0 ]` (empty)

	struct stdlib_ml_sgd_params_float64_params {
	struct stdlib_ml_sgd_float64_params {

Uh oh!

Uh oh!

Conversation

nakul-krishnakumar commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Questions

Other

Checklist

AI Assistance

Disclosure

Uh oh!

nakul-krishnakumar commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nakul-krishnakumar commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kgryte commented Jul 22, 2026

Uh oh!

kgryte commented Jul 22, 2026

Uh oh!

nakul-krishnakumar commented Jul 23, 2026

Uh oh!

nakul-krishnakumar commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nakul-krishnakumar Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

nakul-krishnakumar Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

nakul-krishnakumar Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

nakul-krishnakumar Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

nakul-krishnakumar Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

kgryte Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nakul-krishnakumar commented Jul 22, 2026 •

edited

Loading

nakul-krishnakumar commented Jul 22, 2026 •

edited

Loading

nakul-krishnakumar commented Jul 22, 2026 •

edited

Loading

nakul-krishnakumar commented Jul 23, 2026 •

edited

Loading