Comments (8)
DrDaveD
commented on June 19, 2024
I think you've got the basic idea right. However I would think it would be better for you to generate a .awk script and bindmount that in, rather than making it a pre-awk shell script inside the container. The container doesn't have a mechanism for a shell script to generate awk code. (It is possible for the awk script to escape to shell and do any awk code generation wanted there but I don't think that makes things any easier for you.)
from slate-catalog.
DrDaveD
commented on June 19, 2024
For example, for number 2 you'd do something like
{{ if .Values.SquidConf.Cpu_Affinity_Map }}
setoption("cpu_affinity_map", "{{ .Values.SquidConf.Cpu_Affinity_Map }}")
{{ end }}
from slate-catalog.
LincolnBryant
commented on June 19, 2024
Thanks @DrDaveD !
from slate-catalog.
muali14
commented on June 19, 2024
Hi @DrDaveD,
I am having some difficulty getting the setoption function to work. I added a file "20-slate.awk" under customized.d for testing and its content was:
[root@e9bd680b2c5c squid]# cat customize.d/20-slate.awk
setoption("cpu_affinity_map", "process_numbers=1,2,3,4 cores=2,3,4,5")
[root@e9bd680b2c5c squid]#
But it didn't seem to make a difference. I am not sure if I am suppose to add some additional quotes to above or not?
I tried to run the customize.sh script manually without adding any custom .awk files. The script doesn't return! When I run it in debug mode I get:
$ podman run -it opensciencegrid/frontier-squid:fresh bash
[root@104fa09ecbf7 /]# cd /etc/squid/
[root@104fa09ecbf7 squid]# /bin/bash -x ./customize.sh
++ dirname ./customize.sh
+ cd ./customize.d
+ for f in [0-4]*.sh
+ '[' -f 10-checkvars.sh ']'
+ . 10-checkvars.sh
++ '[' -z '10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 fe80::/10' ']'
++ '[' -z '128 MB' ']'
++ '[' -z 10000 ']'
++ '[' -z /var/cache/squid ']'
++ for f in *.awk
+++ cat 10-stdvars.awk
++ eval 'echo '\''setoption("acl NET_LOCAL src", "'\''$SQUID_IPRANGE'\''")
setoption("cache_mem", "'\''$SQUID_CACHE_MEM'\''")
setoptionparameter("cache_dir", 3, "'\''$SQUID_CACHE_DISK'\''")
setoptionparameter("cache_dir", 2, "'\''$SQUID_CACHE_DISK_LOCATION'\''")'\'''
+++ echo 'setoption("acl NET_LOCAL src", "10.0.0.0/8' 172.16.0.0/12 192.168.0.0/16 fc00::/7 'fe80::/10")
setoption("cache_mem", "128' 'MB")
setoptionparameter("cache_dir", 3, "10000")
setoptionparameter("cache_dir", 2, "/var/cache/squid")'
++ for f in *.awk
+++ cat 90-print.awk
++ eval 'echo '\''print'\'''
+++ echo print
+ awk --file ../customhelps.awk --source '{
setoption("acl NET_LOCAL src", "10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 fe80::/10")
setoption("cache_mem", "128 MB")
setoptionparameter("cache_dir", 3, "10000")
setoptionparameter("cache_dir", 2, "/var/cache/squid")
print
}'
Is the above behavior of the script expected? Isn't it suppose to process pre and post awk scripts and any custom .awk files and then return\exit?
from slate-catalog.
DrDaveD
commented on June 19, 2024
customize.sh is run by systemctl start frontier-squid with /etc/squid/squid.conf.frontierdefault as stdin and squid.conf there as stdout. So yes if you give it no stdin it is expected to hang. It should execute the custom awk files, yes, but I do not see that you bind-mounted in 20-slate.awk into the podman container example.
from slate-catalog.
DrDaveD
commented on June 19, 2024
I don't have podman working but here's what I did with docker:
$ cat 20-slate.awk
setoption("cpu_affinity_map", "process_numbers=1,2,3,4 cores=2,3,4,5")
$ docker run --privileged -v $PWD/20-slate.awk:/etc/squid/customize.d/20-slate.awk opensciencegrid/frontier-squid:fresh bash
[root@dwdosgdev /]# cd /etc/squid
[root@dwdosgdev squid]# ls -l customize.d
total 44
-rw-r--r--. 1 root root 540 Feb 21 01:04 10-checkvars.sh
-rw-r--r--. 1 root root 221 Feb 21 01:04 10-stdvars.awk
-rw-r--r--. 1 3382 9996 71 Feb 22 19:50 20-slate.awk
-rw-r--r--. 1 root root 6 Feb 21 01:04 90-print.awk
[root@dwdosgdev squid]# ./customize.sh <squid.conf.frontierdefault >squid.conf
[root@dwdosgdev squid]# grep ^cpu_affinity squid.conf
cpu_affinity_map process_numbers=1,2,3,4 cores=2,3,4,5
[root@dwdosgdev squid]#
from slate-catalog.
muali14
commented on June 19, 2024
Thanks a lot @DrDaveD
That was very helpful! I did initially mount the .awk file when I was testing on a K8s cluster but I didn't have the file name starts with a numeric character. Since that didn't work I started looking into running the customize.sh script manually and testing outside K8s. I thought only pre and post awk script names need to start with a digit but I was wrong. Adding numeric characters to the .awk file name resolve the issue.
Thanks again!
from slate-catalog.
muali14
commented on June 19, 2024
This has been addressed, and the chart configurations have been normalized against OSG upstream.
from slate-catalog.
Related Issues (20)
- Add Certificate object to HostedCE
- External TLS cert/key not auto-updating for Hosted CE HOT 15
- Version label prevents normal helm upgrades
- Document process for using IGTF certificates with HostedCE
- Investigate using init containers/ container lifecycle hooks
- squid: Ephemeral request should not go up if `CacheDirOnHost: True` HOT 1
- Adding a label selector field to the HostedCE chart that allows an end-user to supply a label selector.
- Integrate with slateci/docker-images HOT 1
- Squid does not respect the memory request in multicore HOT 1
- Squid should accept Disk size for whole node
- Clarify & Simplify logging options
- Incubator/FTS3 Dockerfile does not build HOT 1
- frontier-squid storageClassName is hard coded to "nfs-provisioner" HOT 2
- frontier-squid is a deployment of size 1 and can not be scaled up HOT 6
- frontier-squid requests unnecessary ephemeral-storage when PVCs are used HOT 1
- frontier squid pods may not get scheduled due to nodeselector if PVC is used
- Built index.yaml missing all chart versions HOT 1
- Move FTS3 to Attic
- Update nginx stable app ingress api version to networking.k8s.io/v1
- Update Globus Connect to version 4.0.63 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from slate-catalog.