Regexps in notepad++
With the recent CVE for Roundcube, which even our Polish NASK/CERT sent messages to owners of domains with Roundcube exposed to the Internet, I was in a hurry to patch it.
However, as of pkgsrc 2023Q3 there is no Roundcube 1.6.4, so I did it with the -current branch. This broke my system (luckily it was a clone of one of the mail servers).
Long story short, I was able to update it, but I was left which something like this:
archivers/py-zipp - py311-zipp-3.16.2 missing
comms/py-colorama - py311-colorama-0.4.6 missing
comms/py-rich - py311-rich-13.5.3 missing
converters/py-chardet - py311-chardet-5.2.0 missing
databases/py-aiosqlite - py311-aiosqlite-0.19.0 missing
databases/py-asyncpg - py311-asyncpg-0.28.0 missing
databases/py-multidict - py311-multidict-6.0.4 missing
databases/py-redis - py311-redis-5.0.0 missing
databases/py-sqlite3 - py311-sqlite3-3.11.5nb24 missing
devel/py-aiosignal - py311-aiosignal-1.3.1 missing
devel/py-astroid - py311-astroid-2.15.6 missing
devel/py-async-timeout - py311-async-timeout-4.0.3 missing
devel/py-async_generator - py311-async_generator-1.10nb1 missing
devel/py-attrs - py311-attrs-23.1.0 missing
devel/py-cachetools - py311-cachetools-5.3.1 missing
devel/py-cython - py311-cython-0.29.36 missing
devel/py-distlib - py311-distlib-0.3.7 missing
devel/py-exceptiongroup - py311-exceptiongroup-1.1.3 missing
devel/py-filelock - py311-filelock-3.12.4 missing
devel/py-flit_scm - py311-flit_scm-1.7.0 missing
devel/py-frozenlist - py311-frozenlist-1.4.0 missing
devel/py-hatch-fancy-pypi-readme - py311-hatch-fancy-pypi-readme-23.1.0 missing
devel/py-hatch-vcs - py311-hatch-vcs-0.3.0 missing
devel/py-importlib-metadata - py311-importlib-metadata-6.8.0 missing
devel/py-isort - py311-isort-5.12.0 missing
devel/py-jaraco.classes - py311-jaraco.classes-3.3.0 missing
devel/py-lazy-object-proxy - py311-lazy-object-proxy-1.9.0 missing
devel/py-mccabe - py311-mccabe-0.7.0 missing
devel/py-more-itertools - py311-more-itertools-10.1.0 missing
devel/py-outcome - py311-outcome-1.2.0 missing
devel/py-pep517 - py311-pep517-0.13.0 missing
devel/py-pkginfo - py311-pkginfo-1.9.6 missing
devel/py-poetry-core - py311-poetry-core-1.7.0 missing
devel/py-project-api - py311-project-api-1.6.1 missing
devel/py-py - py311-py-1.11.0 missing
devel/py-pylint - py311-pylint-2.17.5 missing
devel/py-requests-toolbelt - py311-requests-toolbelt-1.0.0 missing
devel/py-sortedcontainers - py311-sortedcontainers-2.4.0nb1 missing
devel/py-tox - py311-tox-4.11.3 missing
devel/py-trio - py311-trio-0.22.2 missing
devel/py-uvloop - py311-uvloop-0.17.0 missing
devel/py-virtualenv - py311-virtualenv-20.24.5 missing
devel/py-wrapt - py311-wrapt-1.15.0 missing
mail/py-postfix-mta-sts-resolver - py311-postfix-mta-sts-resolver-1.4.0 missing
math/py-roman - py311-roman-4.1 missing
misc/py-platformdirs - py311-platformdirs-3.10.0 missing
misc/py-sniffio - py311-sniffio-1.3.0 missing
net/py-aiodns - py311-aiodns-3.0.0 missing
net/py-cares - py311-cares-4.3.0 missing
net/py-twine - py311-twine-4.0.2 missing
security/py-keyring - py311-keyring-24.2.0 missing
security/py-secretstorage - py311-secretstorage-3.3.3 missing
sysutils/py-jeepney - py311-jeepney-0.8.0 missing
textproc/py-dill - py311-dill-0.3.7 missing
textproc/py-docutils - py311-docutils-0.20.1 missing
textproc/py-markdown-it-py - py311-markdown-it-py-3.0.0 missing
textproc/py-pygments - py311-pygments-2.16.1 missing
textproc/py-readme_renderer - py311-readme_renderer-41.0 missing
textproc/py-toml - py311-toml-0.10.2nb1 missing
textproc/py-tomli - py311-tomli-2.0.1 missing
textproc/py-tomlkit - py311-tomlkit-0.12.1 missing
textproc/py-webencodings - py311-webencodings-0.5.1nb1 missing
textproc/py-yaml - py311-yaml-6.0.1 missing
www/py-aiohttp - py311-aiohttp-3.8.5 missing
www/py-bleach - py311-bleach-6.0.0 missing
www/py-mdurl - py311-mdurl-0.1.2 missing
www/py-rfc3986 - py311-rfc3986-2.0.0 missing
www/py-yarl - py311-yarl-1.9.2 missing
This is not a big deal, as it only involves installing missing packages, but it does involve going into each directory under /usr/pkgsrc and doing make install or make update.
As I was in a hurry, I thought it would be quickest to make this list into a simple script that would just cd to the correct directory, then make update, and then go to another directory until the end of the file.
With such a list I had to get rid of everything after - (space, dash, space). Then put cd /usr/pkgsrc/ before each line and finally make update clean every second line.
The final effect should look like this:
cd /usr/pkgsrc/archivers/py-zipp
make update clean
cd /usr/pkgsrc/comms/py-colorama
make update clean
cd /usr/pkgsrc/comms/py-rich
make update clean
cd /usr/pkgsrc/converters/py-chardet
make update clean
cd /usr/pkgsrc/databases/py-aiosqlite
make update clean
cd /usr/pkgsrc/databases/py-asyncpg
make update clean
cd /usr/pkgsrc/databases/py-multidict
make update clean
cd /usr/pkgsrc/databases/py-redis
make update clean
cd /usr/pkgsrc/databases/py-sqlite3
make update clean
cd /usr/pkgsrc/devel/py-aiosignal
make update clean
cd /usr/pkgsrc/devel/py-astroid
make update clean
cd /usr/pkgsrc/devel/py-async-timeout
make update clean
cd /usr/pkgsrc/devel/py-async_generator
make update clean
cd /usr/pkgsrc/devel/py-attrs
make update clean
cd /usr/pkgsrc/devel/py-cachetools
make update clean
cd /usr/pkgsrc/devel/py-cython
make update clean
cd /usr/pkgsrc/devel/py-distlib
make update clean
cd /usr/pkgsrc/devel/py-exceptiongroup
make update clean
cd /usr/pkgsrc/devel/py-filelock
make update clean
cd /usr/pkgsrc/devel/py-flit_scm
make update clean
cd /usr/pkgsrc/devel/py-frozenlist
make update clean
cd /usr/pkgsrc/devel/py-hatch-fancy-pypi-readme
make update clean
cd /usr/pkgsrc/devel/py-hatch-vcs
make update clean
cd /usr/pkgsrc/devel/py-importlib-metadata
make update clean
cd /usr/pkgsrc/devel/py-isort
make update clean
cd /usr/pkgsrc/devel/py-jaraco.classes
make update clean
cd /usr/pkgsrc/devel/py-lazy-object-proxy
make update clean
cd /usr/pkgsrc/devel/py-mccabe
make update clean
cd /usr/pkgsrc/devel/py-more-itertools
make update clean
cd /usr/pkgsrc/devel/py-outcome
make update clean
cd /usr/pkgsrc/devel/py-pep517
make update clean
cd /usr/pkgsrc/devel/py-pkginfo
make update clean
cd /usr/pkgsrc/devel/py-poetry-core
make update clean
cd /usr/pkgsrc/devel/py-project-api
make update clean
cd /usr/pkgsrc/devel/py-py
make update clean
cd /usr/pkgsrc/devel/py-pylint
make update clean
cd /usr/pkgsrc/devel/py-requests-toolbelt
make update clean
cd /usr/pkgsrc/devel/py-sortedcontainers
make update clean
cd /usr/pkgsrc/devel/py-tox
make update clean
cd /usr/pkgsrc/devel/py-trio
make update clean
cd /usr/pkgsrc/devel/py-uvloop
make update clean
cd /usr/pkgsrc/devel/py-virtualenv
make update clean
cd /usr/pkgsrc/devel/py-wrapt
make update clean
cd /usr/pkgsrc/mail/py-postfix-mta-sts-resolver
make update clean
cd /usr/pkgsrc/math/py-roman
make update clean
cd /usr/pkgsrc/misc/py-platformdirs
make update clean
cd /usr/pkgsrc/misc/py-sniffio
make update clean
cd /usr/pkgsrc/net/py-aiodns
make update clean
cd /usr/pkgsrc/net/py-cares
make update clean
cd /usr/pkgsrc/net/py-twine
make update clean
cd /usr/pkgsrc/security/py-keyring
make update clean
cd /usr/pkgsrc/security/py-secretstorage
make update clean
cd /usr/pkgsrc/sysutils/py-jeepney
make update clean
cd /usr/pkgsrc/textproc/py-dill
make update clean
cd /usr/pkgsrc/textproc/py-docutils
make update clean
cd /usr/pkgsrc/textproc/py-markdown-it-py
make update clean
cd /usr/pkgsrc/textproc/py-pygments
make update clean
cd /usr/pkgsrc/textproc/py-readme_renderer
make update clean
cd /usr/pkgsrc/textproc/py-toml
make update clean
cd /usr/pkgsrc/textproc/py-tomli
make update clean
cd /usr/pkgsrc/textproc/py-tomlkit
make update clean
cd /usr/pkgsrc/textproc/py-webencodings
make update clean
cd /usr/pkgsrc/textproc/py-yaml
make update clean
cd /usr/pkgsrc/www/py-aiohttp
make update clean
cd /usr/pkgsrc/www/py-bleach
make update clean
cd /usr/pkgsrc/www/py-mdurl
make update clean
cd /usr/pkgsrc/www/py-rfc3986
make update clean
cd /usr/pkgsrc/www/py-yarl
make update clean
And it was kind of fun to get the regexp right. For convenience I used notepad++ with its find and replace option (CTRL+H) and checked the regexp option there.
The first regexp to get rid of everything after the first space was easy. It's just -.* (note the space before the hyphen).
Why is that?
-matches the characters - literally (case sensitive)
.matches any character (except for line terminators)
* matches the previous token between zero and unlimited times, as many times as possible, giving back as needed (greedy)
So we are left with something like:
Now for the second part, add (or replace in this scenario) cd /usr/pkgsrc/ in front of each line. This is as simple as typing ^ in search and cd /usr/pkgsrc/ in replace:
Why this works?
^ is first character on the line.
So we have this now:
Last but not least - adding defined text every two lines is a bit tricky. Working regexp was:(.*\r?\n){1}\K
And we finally get things sorted:
Now I have copied this output to a file on the NetBSD mail server and just run it with my zsh shell.
And it crawled every path and updated missing packages.