Comments (9)
@michaelmagyar I was able to get the proper permission this week and I changed helper applications to public. So you should be able to see them now.
However, although there is a published ESF package on AWS SAR, the publisher is not verified and the helper/nested applications are missing.
For the publisher not being verified, we are still working with the AWS side to get that solved.
We have not published a new version of ESF yet due to some other issues. Will update this ticket once we are able to publish a new version.
from elastic-serverless-forwarder.
Adding some thoughts as Functionbeat is deprecated with support ending in less than a year, and federal clients and their service providers will need extensive time to plan, implement, re-document, and possibly get re-audited on the change (i.e., they may have to add SQS, Secrets Manager, etc. to their packages if they weren't using them before).
Elastic currently has a GovCloud account cluster that supports the FedRAMP offering. Because Elastic is already approved for GovCloud, the team that handles that account cluster should be able to either add ESF publishing directly to the existing GovCloud account(s) or create a new GovCloud account specifically for ESF publishing. The issue is compliance, and that needs to be thought of first in this case.
It may be possible to immediately add ESF to the existing GovCloud accounts. However, there might be some compliance issues with immediately adding additional functionality to that environment because that might change the ATO package and require a significant change request/re-audit.
If that is going to take some time (it could easily be 6-12 months), then an alternative is to:
- Now: Have a separate GovCloud account provisioned just for publishing ESF to the GovCloud SAR
- Later: Add ESF to the FedRAMP compliance package and migrate it to the existing federal AWS GovCloud accounts
- Even Later: Deprecate the initial GovCloud SAR registry publication and recommend customers switch to the new ARN
If that flow ends up happening and ESF is published to GovCloud outside of the existing federal offering, some entities still may not be able to leverage it for compliance reasons, but they would at least have the ability to make the risk-based decision and/or test that it works with time to implement it properly.
I think the steps are likely:
- Coordinate with the Elastic federal team to determine when ESF will become part of the federal offering (does it require a re-audit or can it be added immediately)
- If not immediately, consider requesting another GovCloud account that will remain outside of the FedRAMP package even if just for temporary ESF publishing to fix any compatibility issues with GovCloud
- Set up the existing pipeline to additionally publish to the chosen GovCloud account
- Tweak the ESF code that is not compatible with GovCloud (e.g., ARNs need to be "arn:${AWS::Partition}:" instead of "arn:aws:")
I hope this happens quickly and that it can be immediately added to the existing FedRAMP accounts/package, but I am not optimistic.
from elastic-serverless-forwarder.
@michaelmagyar I'd need to pair with someone with access to the GovCloud account cluster with the proper permission to publish an app on SAR.
They'll have to publish the forwarder and maintain it on the GovCloud account. From the technical requirements there should not be much to do, but I cannot do anything on my side without pairing with them and transfer them the knowledge they require.
The same if another GovCloud account is requested.
Let's just arrange the proper point of contact and I don't see, from the technical point of view, any great blockers on this.
from elastic-serverless-forwarder.
I see that this issue has not been resolved for almost a year. What is the current plan to support serverless log forwarding in GovCloud? Go Lambda support is being discontinued at the end of this year, so functionbeats will no longer be viable, even as a deprecated option, without a lot of changes.
Is the plan really to force GovCloud clients to add VMs just to collect logs? What about clients that don't currently use/authorize VMs in their environment?
from elastic-serverless-forwarder.
Hey @michaelmagyar, we still have the plan to support serverless log forwarding in GovCloud. We are working with Legal Team to have the Sponsor approval to make it happen.
More info here: https://github.com/elastic/infosec/issues/14266
from elastic-serverless-forwarder.
Hello Elastic. Can you please provide an update to this? I see that PR #510 was merged. However, although there is a published ESF package on AWS SAR, the publisher is not verified and the helper/nested applications are missing.
What is the current status/timeline?
We were told that our current spend of 5 figures a month is not large enough to have access to a technical account manager except for renewals, so this appears to be the only route for us to get updates on this outside of support.
Note: functionbeat is no longer deployable given that AWS has disabled the Go runtime for new functions, so continuing to use that would require rebuilding the package using AL2 runtime and adding Go on top.
from elastic-serverless-forwarder.
Hi @michaelmagyar , we are still waiting for internal permissions to be granted for us, to deploy last ESF version in GovCloud SAR. The ETA is in 2 weeks.
- https://github.com/elastic/enhancements/issues/17413
- https://github.com/elastic/enhancements/issues/18231
from elastic-serverless-forwarder.
Hello @bturquet , are there any updates on the ETA for ESF in GovCloud?
from elastic-serverless-forwarder.
hi @michaelmagyar , we are still blocked on having proper access to GovCloud in order to make ESF available there.
We'll send an update next week
from elastic-serverless-forwarder.
Related Issues (20)
- Incorrect Handling of Permissions Can Result in Data Loss
- Provide support for Python 3.12
- [DOCS] Fix outdated link
- CloudWatch log retention for ESF own logs
- Investigate ingestion of files with same IDs HOT 2
- Create pipeline to push zip file with dependencies to an S3 bucket HOT 3
- Misconfigured ESF input results in a warning message, not events sent to replay queue HOT 3
- CMA-CGM ESF Issue HOT 1
- no input defined on cloudwatch-logs trigger HOT 1
- Enable shared outputs and multiple outputs per input
- Investigate number of API calls of ESF due to ec2:describe_regions usage HOT 3
- Issue with region name check in publishing script
- Support SSM (Systems Manager) parameters
- policy size limits - max 44 log groups you can add
- Deployment method using AWS Console doesn't work HOT 5
- ESF incorrectly sets region for VPC flow logs from multi-region S3 bucket
- es_dead_letter_index
- Sercrets manager secrets caching seems not working
- What do I do when I hit the size limits?
- error: Argument 1 to "wrapper" has incompatible type "ProtocolStorageType@__init__"; expected "ProtocolStorageType@json_collector" [arg-type] HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elastic-serverless-forwarder.