Skip to content

Add msgo/module/hash telemetry counter #2283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gdams wants to merge 2 commits into
base: microsoft/main
Choose a base branch
from
+158 −59
Open

Add msgo/module/hash telemetry counter #2283

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
af45c99
update
gdams May 26, 2026
a747587
review fixes
gdams May 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions eng/doc/Telemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,24 @@ For the blog post announcing this feature, see [Microsoft Go Telemetry](https://

## What is collected

All telemetry data consists of **counters** — simple event names that are incremented.
Telemetry data consists of **counters** (simple event names that are incremented) and **properties** (key-value custom dimensions attached to a counter event).
No source code, file paths, module contents, or personally identifiable information is collected.
Each telemetry session is assigned a random session ID that cannot be linked back to a user.

The counters collected are defined in the [telemetry upload configuration](/go/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/config/config.json) and are listed below.
The counters and properties collected are defined in the [telemetry upload configuration](https://github.com/microsoft/go-infra/blob/main/telemetry/config/config.json) and are listed below.
Comment thread
gdams marked this conversation as resolved.

### Invocation counter

| Counter | Description |
|---|---|
| `go/invocations` | Incremented once per `go` command invocation. |

#### Properties on `go/invocations`

| Property | Description |
|---|---|
| `msgo/module/hash` | A SHA-256 hash of the Go module path from the nearest `go.mod` file. This is a one-way hash — the original module path cannot be recovered from it. Only sent when a `go.mod` file is found in the current directory or a parent directory. |

### Subcommand

| Counter | Description |
Expand Down Expand Up @@ -55,6 +61,7 @@ The counters collected are defined in the [telemetry upload configuration](/go/s

- No source code or file contents.
- No file paths from the local file system.
- No module names or paths — only a one-way SHA-256 hash of the module path is collected.
- No environment variable values (only the presence of specific CI-related variables is checked).
- No network or authentication information.
- No personally identifiable information (PII).
Expand Down
122 changes: 81 additions & 41 deletions patches/0001-Vendor-external-dependencies.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ Use a 'go' that was recently built by the current branch to ensure stable result
.../microsoft/go-infra/telemetry/ci.go | 97 +
.../go-infra/telemetry/config/LICENSE | 21 +
.../go-infra/telemetry/config/config.go | 11 +
.../go-infra/telemetry/config/config.json | 79 +
.../go-infra/telemetry/counter/counter.go | 63 +
.../go-infra/telemetry/config/config.json | 77 +
.../go-infra/telemetry/counter/counter.go | 71 +
.../telemetry/internal/appinsights/README.md | 12 +
.../telemetry/internal/appinsights/client.go | 192 ++
.../telemetry/internal/appinsights/client.go | 200 ++
.../internal/appinsights/inmemorychannel.go | 349 +++
.../internal/contracts/contexttagkeys.go | 50 +
.../internal/contracts/envelope.go | 123 +
Expand All @@ -28,10 +28,10 @@ Use a 'go' that was recently built by the current branch to ensure stable result
.../telemetry/internal/appinsights/package.go | 13 +
.../internal/appinsights/telemetrycontext.go | 44 +
.../internal/appinsights/transmitter.go | 172 ++
.../telemetry/internal/config/config.go | 78 +
.../telemetry/internal/config/config.go | 80 +
.../telemetry/internal/telemetry/proginfo.go | 46 +
.../telemetry/internal/telemetry/telemetry.go | 32 +
.../microsoft/go-infra/telemetry/telemetry.go | 144 +
.../microsoft/go-infra/telemetry/telemetry.go | 168 ++
src/cmd/vendor/modules.txt | 11 +
src/crypto/internal/backend/deps_ignore.go | 22 +
src/go.mod | 6 +
Expand Down Expand Up @@ -271,7 +271,7 @@ Use a 'go' that was recently built by the current branch to ensure stable result
.../internal/subtle/aliasing.go | 32 +
.../internal/sysdll/sys_windows.go | 55 +
src/vendor/modules.txt | 23 +
263 files changed, 35719 insertions(+), 7 deletions(-)
263 files changed, 35759 insertions(+), 7 deletions(-)
create mode 100644 src/cmd/internal/telemetry/counter/deps_ignore.go
create mode 100644 src/cmd/vendor/github.com/microsoft/go-infra/telemetry/LICENSE
create mode 100644 src/cmd/vendor/github.com/microsoft/go-infra/telemetry/README.md
Expand Down Expand Up @@ -529,30 +529,30 @@ Use a 'go' that was recently built by the current branch to ensure stable result
create mode 100644 src/vendor/github.com/microsoft/go-crypto-winnative/internal/sysdll/sys_windows.go

diff --git a/src/cmd/go.mod b/src/cmd/go.mod
index 29b78693abd787..ffcaa3d7848408 100644
index 29b78693abd787..8f3e096d276223 100644
--- a/src/cmd/go.mod
+++ b/src/cmd/go.mod
@@ -4,6 +4,8 @@ go 1.27

require (
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83
+ github.com/microsoft/go-infra/telemetry v0.0.0-20260522114235-13e33af6063a
+ github.com/microsoft/go-infra/telemetry/config v0.0.0-20260522114235-13e33af6063a
+ github.com/microsoft/go-infra/telemetry v0.0.0-20260526160655-aa04f117b3ce
+ github.com/microsoft/go-infra/telemetry/config v0.0.0-20260526160655-aa04f117b3ce
golang.org/x/arch v0.27.1-0.20260521044007-9c1a596a2c97
golang.org/x/build v0.0.0-20260122183339-3ba88df37c64
golang.org/x/mod v0.36.1-0.20260513122029-343ee60345a1
diff --git a/src/cmd/go.sum b/src/cmd/go.sum
index 929e1c34dcbec1..68262698d63570 100644
index 929e1c34dcbec1..e5b03ca050f661 100644
--- a/src/cmd/go.sum
+++ b/src/cmd/go.sum
@@ -4,6 +4,10 @@ github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 h1:z2ogiKUYzX5Is6zr/v
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/ianlancetaylor/demangle v0.0.0-20250417193237-f615e6bd150b h1:ogbOPx86mIhFy764gGkqnkFC8m5PJA7sPzlk9ppLVQA=
github.com/ianlancetaylor/demangle v0.0.0-20250417193237-f615e6bd150b/go.mod h1:gx7rwoVhcfuVKG5uya9Hs3Sxj7EIvldVofAWIUtGouw=
+github.com/microsoft/go-infra/telemetry v0.0.0-20260522114235-13e33af6063a h1:1yIH/VJLOUUWw2yVBEqAqDsakqIsx1TyjHv3rQuVv2A=
+github.com/microsoft/go-infra/telemetry v0.0.0-20260522114235-13e33af6063a/go.mod h1:LxxLUDlqi1gwmGrnh1slAFqnEhZnKjn37MABZ6xJs44=
+github.com/microsoft/go-infra/telemetry/config v0.0.0-20260522114235-13e33af6063a h1:F/bJDa61Hj7ISaUMyErZ9m0GIgHSaLMrhf2f1Pja9Io=
+github.com/microsoft/go-infra/telemetry/config v0.0.0-20260522114235-13e33af6063a/go.mod h1:t6u8QcO4tExYT4+NEB0XqR0ObiUvLe0D8ZpxFobGuA8=
+github.com/microsoft/go-infra/telemetry v0.0.0-20260526160655-aa04f117b3ce h1:mIjWtu9i00ZMtAmGy0gJYvW3P6bqnGt1s1J8aG87UuQ=
+github.com/microsoft/go-infra/telemetry v0.0.0-20260526160655-aa04f117b3ce/go.mod h1:LxxLUDlqi1gwmGrnh1slAFqnEhZnKjn37MABZ6xJs44=
+github.com/microsoft/go-infra/telemetry/config v0.0.0-20260526160655-aa04f117b3ce h1:+9hsuF75XR8yABS2Oo9yevxvqxxS5+58qPH5rwJW4CM=
+github.com/microsoft/go-infra/telemetry/config v0.0.0-20260526160655-aa04f117b3ce/go.mod h1:t6u8QcO4tExYT4+NEB0XqR0ObiUvLe0D8ZpxFobGuA8=
github.com/yuin/goldmark v1.6.0 h1:boZcn2GTjpsynOsC0iJHnBWa4Bi0qzfJjthwauItG68=
github.com/yuin/goldmark v1.6.0/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/arch v0.27.1-0.20260521044007-9c1a596a2c97 h1:OEbDVxixMxnrAI3whhcFkCb0rPrEHwxeSSUMdN0V414=
Expand Down Expand Up @@ -770,10 +770,10 @@ index 00000000000000..044268a046a54d
+var Config []byte
diff --git a/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/config/config.json b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/config/config.json
new file mode 100644
index 00000000000000..89ce84ff435575
index 00000000000000..e2ba3596943583
--- /dev/null
+++ b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/config/config.json
@@ -0,0 +1,79 @@
@@ -0,0 +1,77 @@
+{
+ "GOOS": [
+ "aix",
Expand Down Expand Up @@ -826,7 +826,8 @@ index 00000000000000..89ce84ff435575
+ "Name": "cmd/go",
+ "Counters": [
+ {
+ "Name": "go/invocations"
+ "Name": "go/invocations",
+ "Properties": ["msgo/module/hash"]
+ },
+ {
+ "Name": "go/goexperiment:{ms_tls_config_schannel,systemcrypto,nosystemcrypto,opensslcrypto,cngcrypto,darwincrypto,ms_nocgo_opensslcrypto}"
Expand All @@ -845,9 +846,6 @@ index 00000000000000..89ce84ff435575
+ },
+ {
+ "Name": "msgo/systemcrypto:{enabled,disabled}"
+ },
+ {
+ "Name": "msgo/module:*"
+ }
+ ]
+ }
Expand All @@ -856,10 +854,10 @@ index 00000000000000..89ce84ff435575
\ No newline at end of file
diff --git a/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/counter/counter.go b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/counter/counter.go
new file mode 100644
index 00000000000000..542b9c780b5f1b
index 00000000000000..808caafd45e383
--- /dev/null
+++ b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/counter/counter.go
@@ -0,0 +1,63 @@
@@ -0,0 +1,71 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
Expand Down Expand Up @@ -892,6 +890,14 @@ index 00000000000000..542b9c780b5f1b
+ return telemetry.Client.NewEvent(name, nil)
+}
+
+// NewWithProperties returns a counter with the given name and custom properties.
+// Properties are sent as custom dimensions in Application Insights, allowing
+// high-cardinality values (such as hashes) to be sent as values rather than
+// as part of the counter name.
+func NewWithProperties(name string, properties map[string]string) *Counter {
+ return telemetry.Client.NewEvent(name, properties)
+}
+
+// CountFlags creates a counter for every flag that is set
+// and increments the counter. The name of the counter is
+// the concatenation of prefix and the flag name.
Expand Down Expand Up @@ -943,10 +949,10 @@ index 00000000000000..2e96890f60b871
+- Improve testing.
diff --git a/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/internal/appinsights/client.go b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/internal/appinsights/client.go
new file mode 100644
index 00000000000000..c3dd88924fdcfa
index 00000000000000..b54a026b0fe540
--- /dev/null
+++ b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/internal/appinsights/client.go
@@ -0,0 +1,192 @@
@@ -0,0 +1,200 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
Expand Down Expand Up @@ -1002,6 +1008,11 @@ index 00000000000000..c3dd88924fdcfa
+ // If nil, all telemetry items are sent.
+ UploadFilter func(name string) bool
+
+ // Function to filter property keys for a given event name.
+ // Returns a new map containing only the allowed keys.
+ // If nil, all properties are sent.
+ PropertyFilter func(name string, properties map[string]string) map[string]string
+
+ channel *inMemoryChannel
+ context *telemetryContext
+
Expand Down Expand Up @@ -1103,6 +1114,9 @@ index 00000000000000..c3dd88924fdcfa
+ if n == 0 || (c.UploadFilter != nil && !c.UploadFilter(data.Name)) {
+ return
+ }
+ if c.PropertyFilter != nil && data.Properties != nil {
+ data.Properties = c.PropertyFilter(data.Name, data.Properties)
+ }
+ if !c.init() {
+ // Stop or Close consumed initOnce before init could run.
+ return
Expand Down Expand Up @@ -2005,10 +2019,10 @@ index 00000000000000..0299c87c81cb41
+}
diff --git a/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/internal/config/config.go b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/internal/config/config.go
new file mode 100644
index 00000000000000..1cb6c22a7bcd72
index 00000000000000..e0070ab70977e5
--- /dev/null
+++ b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/internal/config/config.go
@@ -0,0 +1,78 @@
@@ -0,0 +1,80 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
Expand All @@ -2029,16 +2043,18 @@ index 00000000000000..1cb6c22a7bcd72
+
+// A ProgramConfig contains the configuration for a single program.
+type ProgramConfig struct {
+ // the counter names may have to be
+ // repeated for each program. (e.g., if the counters are in a package
+ // that is used in more than one program.)
+ // The counter and property names may have to be repeated for each
+ // program (e.g., if they are defined in a package used by more than
+ // one program).
+ Name string
+ Counters []CounterConfig `json:",omitempty"`
+}
+
+// A CounterConfig contains the configuration for a single counter.
+// Counters may optionally declare associated property event names.
+type CounterConfig struct {
+ Name string // The "collapsed" counter: <chart>:{<bucket1>,<bucket2>,...}
+ Name string // The "collapsed" counter: <chart>:{<bucket1>,<bucket2>,...}
+ Properties []string `json:",omitempty"` // Associated property event names
Comment thread
gdams marked this conversation as resolved.
+}
+
+func ReadConfig(file string) (*UploadConfig, error) {
Expand Down Expand Up @@ -2179,10 +2195,10 @@ index 00000000000000..02d112edb8ad3e
+}
diff --git a/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/telemetry.go b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/telemetry.go
new file mode 100644
index 00000000000000..e50deaa4189b18
index 00000000000000..016de9bdd95956
--- /dev/null
+++ b/src/cmd/vendor/github.com/microsoft/go-infra/telemetry/telemetry.go
@@ -0,0 +1,144 @@
@@ -0,0 +1,168 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
Expand Down Expand Up @@ -2239,8 +2255,9 @@ index 00000000000000..e50deaa4189b18
+ Logger *slog.Logger
+}
+
+var countersToUpload map[string]struct{}
+var eventsToUpload map[string]struct{}
+var wildcardPrefixes []string
+var allowedProperties map[string]map[string]struct{}
+
+// Start initializes telemetry using the specified configuration.
+func Start(cfg Config) {
Expand Down Expand Up @@ -2276,7 +2293,8 @@ index 00000000000000..e50deaa4189b18
+ if progIdx == -1 {
+ return // Program not configured for telemetry
+ }
+ countersToUpload = make(map[string]struct{})
+ eventsToUpload = make(map[string]struct{})
+ allowedProperties = make(map[string]map[string]struct{})
+ wildcardPrefixes = nil
+ for _, c := range uploadConfig.Programs[progIdx].Counters {
+ if c.Name == "" {
Expand All @@ -2288,7 +2306,14 @@ index 00000000000000..e50deaa4189b18
+ wildcardPrefixes = append(wildcardPrefixes, prefix)
+ }
+ } else {
+ countersToUpload[e] = struct{}{}
+ eventsToUpload[e] = struct{}{}
+ if len(c.Properties) > 0 {
+ keys := make(map[string]struct{}, len(c.Properties))
+ for _, p := range c.Properties {
+ keys[p] = struct{}{}
+ }
+ allowedProperties[e] = keys
+ }
+ }
+ }
+ }
Expand All @@ -2302,8 +2327,9 @@ index 00000000000000..e50deaa4189b18
+ "ai.application.ver": ver,
+ "ai.cloud.role": prog,
+ },
+ UploadFilter: uploadFilter,
+ Logger: cfg.Logger,
+ UploadFilter: uploadFilter,
+ PropertyFilter: propertyFilter,
+ Logger: cfg.Logger,
+ })
+}
+
Expand All @@ -2317,7 +2343,7 @@ index 00000000000000..e50deaa4189b18
+}
+
+func uploadFilter(name string) bool {
+ if _, ok := countersToUpload[name]; ok {
+ if _, ok := eventsToUpload[name]; ok {
+ return true
+ }
+ for _, prefix := range wildcardPrefixes {
Expand All @@ -2327,23 +2353,37 @@ index 00000000000000..e50deaa4189b18
+ }
+ return false
+}
+
+func propertyFilter(name string, properties map[string]string) map[string]string {
+ keys, ok := allowedProperties[name]
+ if !ok {
+ return nil
+ }
+ filtered := make(map[string]string, len(keys))
+ for k, v := range properties {
+ if _, ok := keys[k]; ok {
+ filtered[k] = v
+ }
+ }
+ return filtered
+}
diff --git a/src/cmd/vendor/modules.txt b/src/cmd/vendor/modules.txt
index f3ed9c985e7bf5..67e1651ba7e3c5 100644
index f3ed9c985e7bf5..f11ce5085c79ca 100644
--- a/src/cmd/vendor/modules.txt
+++ b/src/cmd/vendor/modules.txt
@@ -16,6 +16,17 @@ github.com/google/pprof/third_party/svgpan
# github.com/ianlancetaylor/demangle v0.0.0-20250417193237-f615e6bd150b
## explicit; go 1.13
github.com/ianlancetaylor/demangle
+# github.com/microsoft/go-infra/telemetry v0.0.0-20260522114235-13e33af6063a
+# github.com/microsoft/go-infra/telemetry v0.0.0-20260526160655-aa04f117b3ce
+## explicit; go 1.25
+github.com/microsoft/go-infra/telemetry
+github.com/microsoft/go-infra/telemetry/counter
+github.com/microsoft/go-infra/telemetry/internal/appinsights
+github.com/microsoft/go-infra/telemetry/internal/appinsights/internal/contracts
+github.com/microsoft/go-infra/telemetry/internal/config
+github.com/microsoft/go-infra/telemetry/internal/telemetry
+# github.com/microsoft/go-infra/telemetry/config v0.0.0-20260522114235-13e33af6063a
+# github.com/microsoft/go-infra/telemetry/config v0.0.0-20260526160655-aa04f117b3ce
+## explicit; go 1.24
+github.com/microsoft/go-infra/telemetry/config
# golang.org/x/arch v0.27.1-0.20260521044007-9c1a596a2c97
Expand Down
Loading
Loading