file_source_delete and create via CLI

[entlein]

collecting logs

time="2025-07-08T18:27:59Z" level=info msg="finished unary call with code Unknown" func=github.com/grpc-ecosystem/go-grpc-middleware/logging/logrus.DefaultMessageProducer file="external/com_github_grpc_ecosystem_go_grpc_middleware/logging/logrus/options.go:220" error="Could not find file source for given name: kubescape.json" fields.time=1.648848ms grpc.code=Unknown grpc.method=RemoveFileSource grpc.service=px.vizier.services.metadata.MetadataFileSourceService grpc.start_time="2025-07-08T18:27:59Z" peer.address="10.42.0.16:36368" span.kind=server system=grpc

[entlein]

laborant@node-01:~$ cat /tmp/tetragon4.json

{"time": "init", "node_name": "init", "type": "process_exec", "payload": {"init": true}}

laborant@dev-machine:honeycluster$ /home/laborant/px_entlein run -f testsource.pxl 
Pixie CLI
*******************************
* ENV VARS
*        PX_CLOUD_ADDR=getcosmic.ai
*******************************
 ✔    Preparing schema 
 ✕    Deploying /tmp/tetragon4.json  ERR: Could not deploy tracepoint
Failed to execute script error=Could not deploy tracepoint

laborant@dev-machine:honeycluster$ cat testsource.pxl

import px
import pxlog
import pxtrace

glob = "/tmp/tetragon4.json"
table = "tetragon4.json"
pxlog.FileSource(glob, table, "4h")

df = px.DataFrame(table)

px.display(df)

in the pem logs:

I20250708 18:37:19.790879 568433 source_connector.cc:35] Initializing source connector: LOG_cd3e5adf-88fc-444b-91ab-5bad2701bf0e
I20250708 18:37:19.790891 568433 stirling.cc:464] Adding info class: [LOG_cd3e5adf-88fc-444b-91ab-5bad2701bf0e/tetragon4.json]
I20250708 18:37:19.790901 568433 stirling.cc:638] FileSourceConnector [cd3e5adf-88fc-444b-91ab-5bad2701bf0e] created in 0.027 ms.
E20250708 18:37:19.825452 19045 source_connector.cc:64] Failed to push data. Message = Table_id 24 doesn't exist.
I20250708 18:37:20.168207 18433 file_source_manager.cc:171] File source[cd3e5adf-88fc-444b-91ab-5bad2701bf0e]::/tmp/tetragon4.json has transitioned PENDING_STATE -> RUNNING_STATE
I20250708 18:37:20.168258 18433 file_source_manager.cc:209] Updating schema for file source
I20250708 18:37:23.031993 18433 exec.cc:97] Queries in flight: 0
I20250708 18:37:23.032084 43369 exec.cc:51] Executing query: id=7ca4ff43-79f9-4bb5-a2e4-176c08976fb6
I20250708 18:37:23.034591 43369 exec.cc:63] Completed query: id=7ca4ff43-79f9-4bb5-a2e4-176c08976fb6
I20250708 18:37:26.011732 19045 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=61 pos=89
I20250708 18:37:19.790601 14750 file_source_manager.cc:90] Registering file source: /tmp/tetragon4.json uuid string=cd3e5adf-88fc-444b-91ab-5bad2701bf0e
I20250708 18:37:19.790764 541011 stirling.cc:617] FileSourceConnector [LOG_cd3e5adf-88fc-444b-91ab-5bad2701bf0e] created in 0.091 ms.
I20250708 18:37:19.790783 541011 source_connector.cc:35] Initializing source connector: LOG_cd3e5adf-88fc-444b-91ab-5bad2701bf0e
I20250708 18:37:19.790799 541011 stirling.cc:464] Adding info class: [LOG_cd3e5adf-88fc-444b-91ab-5bad2701bf0e/tetragon4.json]
I20250708 18:37:19.790858 541011 stirling.cc:638] FileSourceConnector [cd3e5adf-88fc-444b-91ab-5bad2701bf0e] created in 0.081 ms.
E20250708 18:37:19.872442 15567 source_connector.cc:64] Failed to push data. Message = Table_id 24 doesn't exist.
I20250708 18:37:20.564666 14750 file_source_manager.cc:171] File source[cd3e5adf-88fc-444b-91ab-5bad2701bf0e]::/tmp/tetragon4.json has transitioned PENDING_STATE -> RUNNING_STATE
I20250708 18:37:20.565037 14750 file_source_manager.cc:209] Updating schema for file source
I20250708 18:37:21.669550 15567 run_core_stats.cc:88] |534000,0,534000,973170,804756,168414,1,14,39872,0,0,0,0,0,0,0,0,21255,0,17850,12818,26633,33980,56920,87456,237216,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20250708 18:37:25.862056 15567 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=59 pos=89

[entlein]

but it seems to work

laborant@dev-machine:honeycluster$ ../px_entlein run -f show_tetra4.pxl 
Pixie CLI
*******************************
* ENV VARS
*        PX_CLOUD_ADDR=getcosmic.ai
*******************************
Table ID: output
  TIME                                  UUID                                  TIME                            NODE NAME  TYPE          PAYLOAD                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
  2025-07-08 18:42:11.705777 +0000 UTC  2612f350-8d71-45a7-9839-16d03eb0fefc  2025-07-08T11:30:12.230127340Z  cplane-01  process_exec  {"dedup":"f7ebb8f4c3d55fb458f43050bce931c1exec","parent":{"arguments":"-namespace k8s.io -id 83a47d562d80e6b54db1d96dda8474cc3ecd02eb165531fedc7563297366e383 -address /run/k3s/containerd/containerd.sock","auid":4294967295,"binary":"/var/lib/rancher/k3s/data/1b7cbbb3a5b11f87bde410c5b493eddc409ba6809dd5318a80c7cc3eadb26f48/bin/containerd-shim-runc-v2","cwd":"/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/83a47d562d80e6b54db1d96dda8474cc3ecd02eb165531fedc7563297366e383","exec_id":"Y3BsYW5lLTAxOjIxOTIxNTAwMDAwMDA6NDk5NDM=","flags":"procFS auid","in_init_tree":false,"parent_exec_id":"Y3BsYW5lLTAxOjgwMDAwMDAwOjE=","pid":49943,"start_time":"2025-07-08T11:30:02.040127372Z","tid":49943,"uid":0},"process":{"arguments":"-DFOREGROUND","auid":4294967295,"binary":"/usr/sbin/apache2","cwd":"/var/www/html","docker":"5e0bf03b76bf23da68d6d3226aaabf2","exec_id":"Y3BsYW5lLTAxOjIyMDIzNDAwMDAwMDA6NTAyNDI=","flags":"procFS auid","in_init_tree":false,"parent_exec_id":"Y3BsYW5lLTAxOjIxOTIxNTAwMDAwMDA6NDk5NDM=","pid":50242,"pod":{"container":{"id":"containerd://5e0bf03b76bf23da68d6d3226aaabf29c8fb199d31d2dae7837730a319b9b59e","image":{"id":"ghcr.io/k8sstormcenter/webapp@sha256:e323014ec9befb76bc551f8cc3bf158120150e2e277bae11844c2da6c56c0a2b","name":"sha256:c622cf306b94e8a6e7cfd718f048015e033614170f19228d8beee23a0ccc57bb"},"name":"mywebapp-app","pid":1,"start_time":"2025-07-08T11:30:12Z"},"name":"webapp-mywebapp-67965968bb-8j8x9","namespace":"webapp","pod_labels":{"app.kubernetes.io/instance":"webapp","app.kubernetes.io/name":"mywebapp","pod-template-hash":"67965968bb"},"workload":"webapp-mywebapp","workload_kind":"Deployment"},"start_time":"2025-07-08T11:30:12.230127365Z","tid":50242,"uid":0}}  

laborant@dev-machine:honeycluster$ cat show_tetra4.pxl
import px

df = px.DataFrame(table="tetragon4.json")

px.display(df)

[entlein]

laborant@dev-machine:honeycluster$ ../px_entlein run -f delete.pxl

Pixie CLI
*******************************
* ENV VARS
*        PX_CLOUD_ADDR=getcosmic.ai
*******************************
 ✕    Preparing schema  ERR: Could not prepare schema
 ✕    Deploying /tmp/tetragon3.json  ERR: Could not deploy tracepoint
 ✕    Deploying /home/laborant/honeycluster/analysis/file_sourcer/kubescape/kubescape_schema_init.json  ERR: Could not deploy tracepoint
 ✕    Deploying /tmp/tetragon2.json  ERR: Could not deploy tracepoint
 ✔    Deploying /tmp/tetragon4.json 
Failed to execute script error=Could not deploy tracepoint

laborant@dev-machine:honeycluster$ cat delete.pxl

import pxtrace
import pxlog

pxlog.DeleteFileSource("/tmp/tetragon4.json")

Could it be @ddelnano that the syntax should be this one -> answer to self: nopes, it only takes one argument, above seems correct(er)

import pxtrace
import pxlog

glob = "/tmp/tetragon4.json"
table = "tetragon4.json"
pxlog.DeleteFileSource(glob, table)

logs in pem

I20250708 18:45:17.439760 19045 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=4761 pos=6194
I20250708 18:45:27.425673 19045 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=4861 pos=6194
I20250708 18:45:30.623009 578418 stirling.cc:721] FileSource [cd3e5adf-88fc-444b-91ab-5bad2701bf0e]: Removed file polling 0.043 ms.
I20250708 18:45:30.719120 18433 file_source_manager.cc:171] File source[cd3e5adf-88fc-444b-91ab-5bad2701bf0e]::/tmp/tetragon4.json has transitioned RUNNING_STATE -> TERMINATED_STATE
I20250708 18:45:51.196008 19045 run_core_stats.cc:88] |500000,0,500000,989067,817812,171255,1,14,40619,0,0,0,0,0,0,0,0,16118,0,17072,11644,22285,29596,47124,67439,248103,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20250708 18:45:51.297210 19045 run_core_stats.cc:111] |main_loop_iters,no_work_iters,useful_iters,push+transfer,transfer,push,min_push+transfer,max_push+transfer,total_0.00_ms,total_0.01_ms,total_0.02_ms,total_0.03_ms,total_0.06_ms,total_0.10_ms,total_0.18_ms,total_0.32_ms,total_0.56_ms,total_1.00_ms,total_1.78_ms,total_3.16_ms,total_5.62_ms,total_10.00_ms,total_17.78_ms,total_31.62_ms,total_56.23_ms,total_100.00_ms,total_177.83_ms,total_316.23_ms,total_562.34_ms,total_1000.00_ms,total_10000000.00_ms,no_work_0.00_ms,no_work_0.01_ms,no_work_0.02_ms,no_work_0.03_ms,no_work_0.06_ms,no_work_0.10_ms,no_work_0.18_ms,no_work_0.32_ms,no_work_0.56_ms,no_work_1.00_ms,no_work_1.78_ms,no_work_3.16_ms,no_work_5.62_ms,no_work_10.00_ms,no_work_17.78_ms,no_work_31.62_ms,no_work_56.23_ms,no_work_100.00_ms,no_work_177.83_ms,no_work_316.23_ms,no_work_562.34_ms,no_work_1000.00_ms,no_work_10000000.00_ms
I20250708 18:46:50.242842 19045 socket_trace_connector.cc:817] ConnTracker statistics: kTotal=541 kReadyForDestruction=4 kCreated=375033 kDestroyed=374492 kDestroyedGens=84847 kProtocolUnknown=449 kProtocolHTTP=69 kProtocolHTTP2=3 kProtocolMySQL=1 kProtocolCQL=0 kProtocolPGSQL=1 kProtocolDNS=1 kProtocolRedis=0 kProtocolNATS=13 kProtocolMongo=0 kProtocolKafka=0 kProtocolMux=0 kProtocolAMQP=0 kProtocolTLS=0 
I20250708 18:46:50.243091 19045 socket_trace_connector.cc:818] SocketTracer statistics: kLossSocketDataEvent=0 kLossSocketControlEvent=0 kLossConnStatsEvent=0 kLossMMapEvent=0 kLossHTTP2Event=0 kLossGrpcCEvent=0 kLossGrpcCHeaderEvent=0 kLossGrpcCCloseEvent=0 kPollSocketDataEventCount=392747 kPollSocketDataEventAttrSize=34561736 kPollSocketDataEventDataSize=100477100 kPollSocketDataEventSize=136330396 kDroppedSocketDataEvent=0 
I20250708 18:47:13.212807 19045 run_core_stats.cc:88] |501000,0,501000,991119,819366,171753,1,14,40701,0,0,0,0,0,0,0,0,16127,0,17090,11657,22303,29620,47134,67461,248907,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
I20250708 18:45:06.614691 15567 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=4659 pos=89
I20250708 18:45:16.612958 15567 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=4759 pos=89
I20250708 18:45:20.394140 15567 socket_trace_connector.cc:817] ConnTracker statistics: kTotal=225 kReadyForDestruction=0 kCreated=398676 kDestroyed=398451 kDestroyedGens=74163 kProtocolUnknown=169 kProtocolHTTP=47 kProtocolHTTP2=1 kProtocolMySQL=0 kProtocolCQL=0 kProtocolPGSQL=1 kProtocolDNS=4 kProtocolRedis=0 kProtocolNATS=3 kProtocolMongo=0 kProtocolKafka=0 kProtocolMux=0 kProtocolAMQP=0 kProtocolTLS=0 
I20250708 18:45:20.394205 15567 socket_trace_connector.cc:818] SocketTracer statistics: kLossSocketDataEvent=0 kLossSocketControlEvent=0 kLossConnStatsEvent=0 kLossMMapEvent=356 kLossHTTP2Event=0 kLossGrpcCEvent=0 kLossGrpcCHeaderEvent=0 kLossGrpcCCloseEvent=0 kPollSocketDataEventCount=280556 kPollSocketDataEventAttrSize=24688928 kPollSocketDataEventDataSize=110506121 kPollSocketDataEventSize=136045656 kDroppedSocketDataEvent=0 
I20250708 18:45:26.608464 15567 file_source_connector.cc:261] Reached EOF for file=/host/tmp/tetragon4.json eof count=4859 pos=89
I20250708 18:45:30.623302 550448 stirling.cc:721] FileSource [cd3e5adf-88fc-444b-91ab-5bad2701bf0e]: Removed file polling 0.037 ms.
I20250708 18:45:31.003551 14750 file_source_manager.cc:171] File source[cd3e5adf-88fc-444b-91ab-5bad2701bf0e]::/tmp/tetragon4.json has transitioned RUNNING_STATE -> TERMINATED_STATE

However, the table is still displaying data

laborant@dev-machine:honeycluster$ ../px_entlein run -f show_tetra4.pxl 
Pixie CLI
*******************************
* ENV VARS
*        PX_CLOUD_ADDR=getcosmic.ai
*******************************
Table ID: output
  TIME                                  UUID                                  TIME                            NODE NAME  TYPE          PAYLOAD                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
  2025-07-08 18:42:11.705777 +0000 UTC  2612f350-8d71-45a7-9839-16d03eb0fefc  2025-07-08T11:30:12.230127340Z  cplane-01  process_exec  {"dedup":"f7ebb8f4c3d55fb458f43050bce931c1exec","parent":{"arguments":"-namespace k8s.io -id 83a47d562d80e6b54db1d96dda8474cc3ecd02eb165531fedc7563297366e383 -address /run/k3s/containerd/containerd.sock","auid":4294967295,"binary":"/var/lib/rancher/k3s/data/1b7cbbb3a5b11f87bde410c5b493eddc409ba6809dd5318a80c7cc3eadb26f48/bin/containerd-shim-runc-v2","cwd":"/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/83a47d562d80e6b54db1d96dda8474cc3ecd02eb165531fedc7563297366e383","exec_id":"Y3BsYW5lLTAxOjIxOTIxNTAwMDAwMDA6NDk5NDM=","flags":"procFS auid","in_init_tree":false,"parent_exec_id":"Y3BsYW5lLTAxOjgwMDAwMDAwOjE=","pid":49943,"start_time":"2025-07-08T11:30:02.040127372Z","tid":49943,"uid":0},"process":{"arguments":"-DFOREGROUND","auid":4294967295,"binary":"/usr/sbin/apache2","cwd":"/var/www/html","docker":"5e0bf03b76bf23da68d6d3226aaabf2","exec_id":"Y3BsYW5lLTAxOjIyMDIzNDAwMDAwMDA6NTAyNDI=","flags":"procFS auid","in_init_tree":false,"parent_exec_id":"Y3BsYW5lLTAxOjIxOTIxNTAwMDAwMDA6NDk5NDM=","pid":50242,"pod":{"container":{"id":"containerd://5e0bf03b76bf23da68d6d3226aaabf29c8fb199d31d2dae7837730a319b9b59e","image":{"id":"ghcr.io/k8sstormcenter/webapp@sha256:e323014ec9befb76bc551f8cc3bf158120150e2e277bae11844c2da6c56c0a2b","name":"sha256:c622cf306b94e8a6e7cfd718f048015e033614170f19228d8beee23a0ccc57bb"},"name":"mywebapp-app","pid":1,"start_time":"2025-07-08T11:30:12Z"},"name":"webapp-mywebapp-67965968bb-8j8x9","namespace":"webapp","pod_labels":{"app.kubernetes.io/instance":"webapp","app.kubernetes.io/name":"mywebapp","pod-template-hash":"67965968bb"},"workload":"webapp-mywebapp","workload_kind":"Deployment"},"start_time":"2025-07-08T11:30:12.230127365Z","tid":50242,"uid":0}}  

and its still in schemata

laborant@dev-machine:honeycluster$ ../px_entlein run px/schemas
Pixie CLI
*******************************
* ENV VARS
*        PX_CLOUD_ADDR=getcosmic.ai
*******************************
Table ID: Pixie Datasource Table Descriptions
  TABLE NAME         TABLE DESC                                                                                                                                                                                                                                                                            TABLE METADATA                        
  mux_events         Mux request-response pair events                                                                                                                                                                                                                                                                                            
  nats_events.beta   NATS messages.                                                                                                                                                                                                                                                                                                              
  tls_events         TLS request-response pair events                                                                                                                                                                                                                                                                                            
  kubescape.json                                                                                                                                                                                                                                                                                           0f29ba30-490a-4620-9aa5-14cda0a782f9  
  proc_exit_events   Traces all abnormal process exits                                                                                                                                                                                                                                                                                           
  stirling_error     This table contains the status of tracepoints in different Stirling source connectors and the error messages.                                                                                                                                                                                                               
  tetragon4.json                                                                                                                                                                                                                                                                                           cd3e5adf-88fc-444b-91ab-5bad2701bf0e  
  pgsql_events       Postgres (pgsql) request-response pair events     

[entlein]

FileSourceCreate via CLI works like a charm 🥇

[ddelnano]

@entlein in the px run -f delete.pxl output from this comment, I see that the other tracepoints are having issues (/tmp/tetragon2.json and /tmp/tetragon3.json). The current mutation handling fails when any mutation is in a bad state, so it appears your cluster had an issue before you tried to issue the deletion.

✕ Deploying /tmp/tetragon3.json ERR: Could not deploy tracepoint
✕ Deploying /home/laborant/honeycluster/analysis/file_sourcer/kubescape/kubescape_schema_init.json ERR: Could not deploy tracepoint
✕ Deploying /tmp/tetragon2.json ERR: Could not deploy tracepoint

I recommend re-testing with a clean metadata state. Starting fresh requires deleting the metadata PVC, restarting the metadata service before trying the file source creation and deletion from scratch.