Comments (20)
I think this should be the order:
- Prove out in a prototype that we can get what we need from the Aws::Pricing:: Client module.
- Upgrade the AWS provider to use the Aws-Sdk V3 ( we need this anyway since V2 was released in 2015 and we should not fall so far behind)
- Update the AWS Refresh code to gather instance types like we gather other inventory types
- If any special configuration is needed on the IAM roles then consider a documentation update.
from manageiq-providers-amazon.
Related issue: #302
from manageiq-providers-amazon.
@AlexanderZagaynov Seems it's partially done. What's the current status? Also, the two doc links appear to be dead.
from manageiq-providers-amazon.
@AlexanderZagaynov FYI in version 3 of the AWS SDK for Ruby, there's an Aws::Pricing::Client
you can use that will simplify your life.
from manageiq-providers-amazon.
@djberg96 @Ladas it doesn't makes sense:
- it requires credentials and IAM role setup
- it returns the same data, as from those JSON files, but filtered (and we don't need any filters there)
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/using-pelong.html
from manageiq-providers-amazon.
@AlexanderZagaynov - Why do we not need filters? If we filter on "Compute Instance" we do not have to pull a ton of data we otherwise would have to pull.
I was able to get the product information without modifying my AWS credentials because I have admin access. We need to find out what roles users are given access to today when setting up the ManageIQ user in AWS. Enabling an IAM role should not stop us from using the Aws::Pricing:: Client module.
from manageiq-providers-amazon.
@AlexanderZagaynov IAM role is ok, since we already require like 10-20 roles (if you don't have admin)
If it's returning the same data, I would rather go with the API and refresh approach, since then it just stays fresh for the customer automatically. While now we need to release for every new flavor, or customer needs to type it in manually to settings. So the experience with refresh is always better.
from manageiq-providers-amazon.
@Ladas the thing was just to replace one parser to another, and not define values in the code file, but in the yaml instead. What you asking about - is to change entire approach, which is definitely out of the current PR's scope.
from manageiq-providers-amazon.
@AlexanderZagaynov if the current approach is working well enough and we might be able to get the flavors from the API (which we weren't able to do before) I agree with @Ladas getting flavors from the API would be preferable even if it is out of the scope of this PR.
from manageiq-providers-amazon.
hi All
FYI Alex and I discussed this offline. He is going to make a new PR using the Aws::Pricing::Client module where he will process instance types like we process all other inventory types.
from manageiq-providers-amazon.
@agrare @bronaghs (cc @Fryguy) we discussed it with @Ladas right now, and make conclusion that there should be at least two PRs within another BZ/issue:
- upgrade
aws-sdk
gem to v3 - refactor
Inventory::Parser
to not use hardcoded instance types (flavors
), but fetch them dynamically during the regular refresh
I'm going to simplify my current PR to just move hardcoded constants to the yaml
file, and update it with a new flavors. It should be enough for now, considering further changes.
from manageiq-providers-amazon.
@AlexanderZagaynov - why do we still need your current PR if we use the AWS SDK?
from manageiq-providers-amazon.
Only reason would be that we need something until we actually upgrade to v3. Unless we're going to require v3 specifically for the rake task, but that might get confusing. But, I'll leave that up to y'all to resolve.
from manageiq-providers-amazon.
@bronaghs updating the current yaml file would make sense, if we would be backporting it farther than the solution using the client v3? So, what are the backport plans? :-)
from manageiq-providers-amazon.
@AlexanderZagaynov @djberg96 @bronaghs FYI, in topology inventory I am using the query RedHatInsights/topological_inventory-amazon@9160963#diff-937a6d3aaf6ea0718b23227a3e3a2a6fR27
Since the JSON has over 100k flavor objects in it, i'd be huge in memory and we'd need to fetch 500MB or so. With a query, I can filter it down to 164 unique instance types now. So definitely +1 for the query approach as @djberg96 said
from manageiq-providers-amazon.
This issue has been automatically marked as stale because it has not been updated for at least 6 months.
If you can still reproduce this issue on the current release or on master
, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions!
from manageiq-providers-amazon.
@AlexanderZagaynov, @Ladas Now that we've upgraded to v3, what's the word?
from manageiq-providers-amazon.
@djberg96 topological inventory is using the API for a while and it works ok (filtering out just what we need)
from manageiq-providers-amazon.
This issue has been automatically marked as stale because it has not been updated for at least 3 months.
If you can still reproduce this issue on the current release or on master
, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions! More information about the ManageIQ triage process can be found in the traige process documentation.
from manageiq-providers-amazon.
from manageiq-providers-amazon.
Related Issues (20)
- Gather AWS instance types list from AWS's GitHub repo HOT 3
- Raise exceptions if SSH commands failed in docker running image HOT 1
- Os on AWS not working HOT 4
- Allow AWS provider to be defined without secret/key in order to assume instance credentials HOT 4
- Upgrade AWS SDK library to version 3 HOT 11
- After switching to aws-sdk v3 paged results are not handled HOT 4
- ManagerMixin::ClassMethods::verify_credentials always returns true HOT 2
- The new `create_from_params` doesn't create an endpoint relation, just the authentication HOT 1
- Are relationships between cloud_networks and network_routers correct? HOT 3
- AWS_EC2_Instance_DELETE events are not always received HOT 8
- AWS Network(VPC) in CFME has no relationship with Network Router(Route Table) HOT 1
- AWS VPC Security Groups and Network Ports(Network Interfaces) don't use in CFME Name field for naming HOT 2
- EC2 Volume size change is not detected by targeted refresh HOT 2
- Amazon SSA not working HOT 4
- [RFE] Support Amazon EKS HOT 1
- Refresh fails normalizing OS name for image with no image_location
- Add a new region to regions.yml HOT 2
- Allow instance types to be pulled from github.com HOT 2
- Verify access to SQS before starting the EventCatcher HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from manageiq-providers-amazon.