Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Documentation/Glue Example #1591

Open
1 task done
nick-at-UW opened this issue Dec 19, 2024 · 3 comments
Open
1 task done

[FEATURE] Documentation/Glue Example #1591

nick-at-UW opened this issue Dec 19, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@nick-at-UW
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Feature description

The documentation states that Glue metrics are available to be pulled via discovery, however I've been unable to get metrics from the Glue jobs. I've attempted to configure this via discovery job as well as customNamespace. For reference, the glue metrics we see do appear to be in a custom namespace called 'Glue'

I've provided a snippet of our config for reference. Any guidance in this configuration would be appreciated.

What might the configuration look like?

    customNamespace:
      - name: Glue
        namespace: Glue
        regions:
          - us-west-2
        metrics:
          - name: glue.JobRunsSucceeded
            statistics: [Sum]
            period: 300
            length: 600
            delay: 300
            nilToZero: true
          - name: glue.JobRunsFailed
            statistics: [Sum]
            period: 300
            length: 600
            delay: 300
            nilToZero: true
    discovery:
      jobs:
        - type: Glue
          regions:
            - us-west-2
          metrics:
            - name: glue.JobRunsSucceeded
              statistics: [Sum]
              period: 300
              length: 600
              delay: 300
              nilToZero: true
            - name: glue.JobRunsFailed
              statistics: [Sum]
              period: 300
              length: 600
              delay: 300
              nilToZero: true

Anything else?

No response

@nick-at-UW nick-at-UW added the enhancement New feature or request label Dec 19, 2024
@asaednu
Copy link

asaednu commented Jan 10, 2025

@nick-at-UW were you able to get Glue metrics working with YACE ?

I'm having the same issue where the only metric YACE has been able to export is aws_glue_info

@nick-at-UW
Copy link
Author

nick-at-UW commented Jan 10, 2025 via email

@asaednu
Copy link

asaednu commented Jan 14, 2025

@nick-at-UW I was able to get this to work. The trick was to not use customNamespace. All you need is the jobs section.

Also the name of the metrics were glue.succeed.ALL and glue.error.ALL. I used Cloudwatch Metrics in AWS Console to figure out the name of the metrics.

So here is the snippet that works:

discovery:
  jobs:
    - type: Glue
      regions:
        - us-east-1
      addCloudwatchTimestamp: true
      period: 60
      length: 300
      delay: 60
      metrics:
        - name: glue.succeed.ALL
          statistics:
            - Sum
          nilToZero: true
        - name: glue.error.ALL
          statistics:
            - Sum
          nilToZero: true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants