Usage charging deleted volumes

[matheusfontes]

ISSUE TYPE

• Bug Report

COMPONENT NAME

usage

CLOUDSTACK VERSION

4.16 and 4.17

CONFIGURATION

N/A

OS / ENVIRONMENT

centos 7

SUMMARY

Usage never set volume as deleted and when volume is resized, it is charge twice (or more)

STEPS TO REPRODUCE

Cenario 1:
Create a volume and attach it on vm.
Remove volume.
It will be charged forever

Cenario 2:
Create a volume and attach it on vm.
Resive volume.
It will be charged for many times that you resize it

EXPECTED RESULTS

Stop to charge volume when it is removed.

ACTUAL RESULTS

Volume is being charged when it is removed or being charged more than 24 hours.

# > list usagerecords domainid=8e1c4859-9cf4-4414-a218-160b05b9f157 accountid=3e64effb-12a6-4642-95f9-c217c804a3c4 type=6 startdate=2020-01-19 enddate=2020-01-19
    {
      "account": "21150",
      "accountid": "3e64effb-12a6-4642-95f9-c217c804a3c4",
      "description": "Volume usage for DATA-600 (40f8412b-65f5-4c2d-a6e3-00a4e0456279) with disk offering Disco Magnético (34573d38-2dce-4d8b-a0b4-80ad61b1cbc8) and size (1000.00 GB) 1073741824000",
      "domain": "skycloud.prv",
      "domainid": "8e1c4859-9cf4-4414-a218-160b05b9f157",
      "enddate": "2020-01-19'T'23:59:59-03:00",
      "offeringid": "34573d38-2dce-4d8b-a0b4-80ad61b1cbc8",
      "rawusage": "48",
      "size": 1073741824000,
      "startdate": "2020-01-19'T'00:00:00-03:00",
      "tags": [],
      "usage": "48 Hrs",
      "usageid": "40f8412b-65f5-4c2d-a6e3-00a4e0456279",
      "usagetype": 6,
      "zoneid": "74c526f8-52a2-4e2e-8d08-76eae35913a4"
    },

[DaanHoogland]

@matheusfontes , in your scenario 1:

Cenario 1:
Create a volume and attach it on vm.
Remove volume.
It will be charged forever

the volume is supposed to be continually charged for, if it is only removed from the VM. Do you mean it is deleted and expunged and than still charged for?

Scenario 2 seems obviously wrong, I'll look into that.

[matheusfontes]

@DaanHoogland volumes expunged are always beeing charged.
I saw some queries in database that I think they are wrong.
When usage job is started and it's looking for a VOLUME.DELETE event it tries to search this volume in usage_volume table with that query:
SELECT usage_volume.id, usage_volume.zone_id, usage_volume.account_id, usage_volume.domain_id, usage_volume.volume_id, usage_volume.disk_offering_id, usage_volume.template_id, usage_volume.size, usage_volume.created, usage_volume.deleted FROM usage_volume WHERE usage_volume.account_id = 9 AND usage_volume.id = 110 AND usage_volume.deleted IS NULL

I think this usage_volume.id search criteria is wrong, it should search for a usage_volume.volume_id field.

[matheusfontes]

Apparently these 2 lines in UsageManagerImpl.java solves the usage volumes problem.
But I think there is a problem with firewall rules and lb usage also.
Ps: Tested on 4.17.0.1 version

UsageManagerImpl.txt

[DaanHoogland]

Can you submit a PR with those lines @matheusfontes ?

[DaanHoogland]

never mind @matheusfontes , found a few secs to do it: #6737

[matheusfontes]

@DaanHoogland now I have sure that the problem is bigger than only resized/deleted volumes.
usage is charging a deleted vrouters networks transfer, deleted firewall rules and load balance.
I need to open other issue? Everyone are experiencing these charges problems?
The problem started on 4.16, so I think everyone that uses usage to billing clients are billing them wrong and this is need a urgent fix, don't it?

[DaanHoogland]

@matheusfontes I think that if these are fixed by separate changes they are seperate issues. If we piut the fix in one patch we can just rename the issue.