new compiler error re: SET-DISPATCH-MACRO-CHARACTER

[Konstanski, Carlos]

This is a bug report. I'm using puri as an example, but it's not a puri
bug.

With the latest version of asdf (3.3.0) puri no longer compiles. The
following compiler error is thrown:

SET-DISPATCH-MACRO-CHARACTER would modify the standard readtable.
[Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]

In:

0: (SET-DISPATCH-MACRO-CHARACTER #\# #\u #<FUNCTION PURI::SHARP-U>
#<READTABLE {100002D6C3}>)

The code:

(defun sharp-u (stream chr arg)
(declare (ignore chr arg))
(let ((arg (read stream nil nil t)))
(if *read-suppress*
nil
(if* (stringp arg)
then (parse-uri arg)
else
(internal-reader-error
stream
"#u takes a string or list argument: ~s" arg)))))

(set-dispatch-macro-character #\# #\u #'puri::sharp-u)

What puri is doing re: SET-DISPATCH-MACRO-CHARACTER is totally by the
book. Why is this suddenly an error? Is there a workaround?

Backing up to the next most recent release of asdf makes the problem go
away:

dev-lisp/asdf-3.2.1-r1:0/3.2.1-r1
dev-lisp/uiop-3.2.1:0

I tried to find the asdf changelog to see if this is a documented
change. But the link is broken.
https://common-lisp.net/project/asdf/Changelog

uiop seems to be closely tied to asdf. Not sure which package is
actually at fault.
--
Carlos Konstanski

[Faré]

I just tried compiling puri with asdf 3.3.0, using either quicklisp or
the latest commit (b537e93 from August 29 2015) on sbcl 1.3.20, and
had no issue whatsoever.

Odds are the problem is on your side, probably with your using
with-standard-io-syntax or some such. Note that it is very rude to use
set-dispatch-macro-character on a readtable you didn't specifically
setup, especially since it might be a read-only readtable, or worse, a
writable one used by people who assume it won't be modified.

I've had a branch of ASDF, called "syntax-control", supposed to bring
*some* sanity in readtables used while building with ASDF. It has
bitrotten in the last 4 years and is nowhere near being merged at this
point.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Opportunity is missed by most people because it comes dressed in overalls
and looks like work. — T. A. Edison


On Wed, Oct 11, 2017 at 8:41 PM, Konstanski, Carlos

Post by Konstanski, Carlos
This is a bug report. I'm using puri as an example, but it's not a puri
bug.
With the latest version of asdf (3.3.0) puri no longer compiles. The
SET-DISPATCH-MACRO-CHARACTER would modify the standard readtable.
[Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]
0: (SET-DISPATCH-MACRO-CHARACTER #\# #\u #<FUNCTION PURI::SHARP-U>
#<READTABLE {100002D6C3}>)
(defun sharp-u (stream chr arg)
(declare (ignore chr arg))
(let ((arg (read stream nil nil t)))
(if *read-suppress*
nil
(if* (stringp arg)
then (parse-uri arg)
else
(internal-reader-error
stream
"#u takes a string or list argument: ~s" arg)))))
(set-dispatch-macro-character #\# #\u #'puri::sharp-u)
What puri is doing re: SET-DISPATCH-MACRO-CHARACTER is totally by the
book. Why is this suddenly an error? Is there a workaround?
Backing up to the next most recent release of asdf makes the problem go
dev-lisp/asdf-3.2.1-r1:0/3.2.1-r1
dev-lisp/uiop-3.2.1:0
I tried to find the asdf changelog to see if this is a documented
change. But the link is broken.
https://common-lisp.net/project/asdf/Changelog
uiop seems to be closely tied to asdf. Not sure which package is
actually at fault.
--
Carlos Konstanski

[Konstanski, Carlos]

puri was pulled into my system by quicklisp as s dependency of drakma, and
asdf was installed by portage. I have never written a line of lisp that
called puri directly. No one is ever persuaded by the "it works for me"
argument. The issue is real. Are you using a clean install of asdf, uiop
and puri? Or are you using your development branch of asdf? Developers can
always get it to work on their workstations. Maybe it's the gentoo package
for asdf or uiop. Maybe slime is providing some sort of wrapping
environment that affects readtables. There are any number of things that
could be at play that are beyond the user's control.

Let's say you are right in saying that puri is doing something naughty by
polluting a readtable. I'll go along with that. I have no reason to doubt
it. I'm on your side on this issue. Nevertheless we have a problem: this
code has been in the wild since 2015 or earlier. A very ubiquitous library
(drakma) depends on it. Cleaning up poor behavior is a bigger job than
making a fundamental and core component like asdf no longer accept the
behavior. The offending libraries have to be fixed in concert with this
refactoring. Established libraries cannot simply have the rug pulled out
from under them.

I'm with Stas. I cannot upgrade asdf if it makes it impossible for me to
use libraries that I depend on. Whatever new things asdf does are less
important to me than having a working drakma. I believe just about all
users would agree with this sentiment.

I'm perfectly willing to contribute to the fixing of these libraries. But
let's not do it as a mad scramble. Let's issue deprecation warnings,
identify broken quicklisp packages, get the bugs in those packages fixed
and then implement the new asdf behavior. I'm here to help.

Carlos

Post by Faré
I just tried compiling puri with asdf 3.3.0, using either quicklisp or
the latest commit (b537e93 from August 29 2015) on sbcl 1.3.20, and
had no issue whatsoever.
Odds are the problem is on your side, probably with your using
with-standard-io-syntax or some such. Note that it is very rude to use
set-dispatch-macro-character on a readtable you didn't specifically
setup, especially since it might be a read-only readtable, or worse, a
writable one used by people who assume it won't be modified.
I've had a branch of ASDF, called "syntax-control", supposed to bring
*some* sanity in readtables used while building with ASDF. It has
bitrotten in the last 4 years and is nowhere near being merged at this
point.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics•
https://urldefense.proofpoint.com/v2/url?u=http-3A__fare.
tunes.org&d=DwIFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LR
xpb6__0PomBTQ&r=UXttlWX_R8MsWhBAnFwO2t5aPuH2x7HGk4gZpT
TCo-UE-RFLppFABNHDw29AmFnQ&m=sNgypPrTU-ocowWcZQAmsxdniF2mxBwl_
6BmfJxiyjY&s=8O9az24Gmg12t9C9jp5bOiJSXjwPqkIvmeFuYqIUCnw&e=
Opportunity is missed by most people because it comes dressed in overalls
and looks like work. — T. A. Edison
On Wed, Oct 11, 2017 at 8:41 PM, Konstanski, Carlos

Post by Konstanski, Carlos
This is a bug report. I'm using puri as an example, but it's not a puri
bug.
With the latest version of asdf (3.3.0) puri no longer compiles. The
SET-DISPATCH-MACRO-CHARACTER would modify the standard readtable.
[Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]
0: (SET-DISPATCH-MACRO-CHARACTER #\# #\u #<FUNCTION PURI::SHARP-U>
#<READTABLE {100002D6C3}>)
(defun sharp-u (stream chr arg)
(declare (ignore chr arg))
(let ((arg (read stream nil nil t)))
(if *read-suppress*
nil
(if* (stringp arg)
then (parse-uri arg)
else
(internal-reader-error
stream
"#u takes a string or list argument: ~s" arg)))))
(set-dispatch-macro-character #\# #\u #'puri::sharp-u)
What puri is doing re: SET-DISPATCH-MACRO-CHARACTER is totally by the
book. Why is this suddenly an error? Is there a workaround?
Backing up to the next most recent release of asdf makes the problem go
dev-lisp/asdf-3.2.1-r1:0/3.2.1-r1
dev-lisp/uiop-3.2.1:0
I tried to find the asdf changelog to see if this is a documented
change. But the link is broken.
https://urldefense.proofpoint.com/v2/url?u=https-3A__common-

2Dlisp.net_project_asdf_Changelog&d=DwIFaQ&c=
udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=UXttlWX_
R8MsWhBAnFwO2t5aPuH2x7HGk4gZpTTCo-UE-RFLppFABNHDw29AmFnQ&m=sNgypPrTU-
ocowWcZQAmsxdniF2mxBwl_6BmfJxiyjY&s=-Ety3cawXfSBOqWK_
Ms5Hi2TwB62089buk66HVbLu-s&e=

Post by Konstanski, Carlos
uiop seems to be closely tied to asdf. Not sure which package is
actually at fault.
--
Carlos Konstanski

[Faré]

On Wed, Oct 11, 2017 at 11:19 PM, Konstanski, Carlos

Post by Konstanski, Carlos
puri was pulled into my system by quicklisp as s dependency of drakma, and
asdf was installed by portage. I have never written a line of lisp that
called puri directly.

Good for you. Have you written a line of lisp that calls asdf? Because
asdf does not call itself magically after reading your thoughts. Would
perchance your build scripts be using with-standard-io-syntax or some
such?

Post by Konstanski, Carlos
No one is ever persuaded by the "it works for me"
argument.

No one is ever persuaded by the "it doesn't work for me" argument.
<insert copious expletives here>

Post by Konstanski, Carlos
The issue is real.

(realp 'issue) => NIL
Not reproducible. Come back with a reproducible test case and a complete log.
http://www.catb.org/~esr/reposurgeon/reporting-bugs.html

Post by Konstanski, Carlos
Are you using a clean install of asdf, uiop and
puri? Or are you using your development branch of asdf? Developers can
always get it to work on their workstations. Maybe it's the gentoo package
for asdf or uiop. Maybe slime is providing some sort of wrapping environment
that affects readtables. There are any number of things that could be at
play that are beyond the user's control.

You're the one experiencing an issue. These questions are for you to
answer. No one is going to hack into your computer to extract the
configuration information necessary to produce a useful bug report.

Post by Konstanski, Carlos
Let's say you are right in saying that puri is doing something naughty by
polluting a readtable.

No. YOU say it. I'm just interpreting in English the precious little
information that you claim you extracted from a backtrace generated in
mysterious ways that you didn't tell anything about.

Post by Konstanski, Carlos
I'll go along with that. I have no reason to doubt
it. I'm on your side on this issue.

Frankly not only do you sound like you're not on my side, you sound
like you're not even on your own side.
No. YOU have a problem. No one else has a problem.
Before you blame others, what are YOU doing wrong?

Post by Konstanski, Carlos
this
code has been in the wild since 2015 or earlier. A very ubiquitous library
(drakma) depends on it. Cleaning up poor behavior is a bigger job thant
making a fundamental and core component like asdf no longer accept the
behavior. The offending libraries have to be fixed in concert with this
refactoring. Established libraries cannot simply have the rug pulled out
from under them.

Not only puri but all of quicklisp builds fine for me with ASDF 3.3.0,
except a handful of well-identified systems that are unmaintained
(patch not merged and no answer from maintainers after repeated
prodding over 6 months).

Post by Konstanski, Carlos
I'm with Stas. I cannot upgrade asdf if it makes it impossible for me to use
libraries that I depend on. Whatever new things asdf does are less important
to me than having a working drakma. I believe just about all users would
agree with this sentiment.

Drakma works perfectly with ASDF 3.3.0, thank you. (asdf:test-system
:drakma)... Did 40 checks. Pass: 40 (100%).

Post by Konstanski, Carlos
I'm perfectly willing to contribute to the fixing of these libraries. But
let's not do it as a mad scramble. Let's issue deprecation warnings,
identify broken quicklisp packages, get the bugs in those packages fixed and
then implement the new asdf behavior. I'm here to help.

You're being extremely unhelpful --- including to yourself.

What ASDF 3.3.0 does change compared to ASDF 3.2.1 is the behavior
with respect to :defsystem-depends-on libraries (and more generally,
libraries loaded inside a .asd) and their transitive dependencies. No
more double-loading or missed loading. If you were depending on some
double-loading to counteract side-effects to the readtable by some
library, you lose.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
If you make people think they're thinking, they'll love you;
but if you really make them think, they'll hate you. — Don Marquis

[Robert Goldman]

I'm with Faré on this one. I don't see evidence that this change is
because ASDF is doing something bad. I believe it's consistent with the
hypothesis that there was some imperfectly-controlled aspect of building
that is done differently now (e.g., files loaded in a different order
where both orders are compatible with the constraints in the system
definitions).

This doesn't even look like an ASDF error to me -- it looks like an
error that occurred while loading a system that ASDF has re-packaged.

So we might be able to help you debug this, but without more evidence,
there's no reason to believe that ASDF has done anything to you: it
looks like the change in ASDF just reveals a pre-existing bug.

[73budden .]

Warning about modifying standard readtable is issued by SBCL (at least
in my SBCL 1.3.18), please grep for the message in SBCL sources, and
it seem to be introduced in 1.0.24. I took a look at puri's definition
(maybe old, in my local copy of quicklisp) and it looks like it
actually tries to modify standard readtable. But here it does not
happen when it is being loaded via my patched asdf 3.1.6 and my own
IDE with forked SLIME with many systems pre-loaded. I don't know why
and have no time to investigate further. Obviously, my insights would
be of limited use because my environment is very different from that
of OP.

Post by Robert Goldman
So we might be able to help you debug this

It'd be very nice to have instructions on how to trace what happens
while system is being built:that is, which files are compiled, which
are loaded and what is a readtable in the beginning of each load
operation.

Running builds in old and new asdf versions and comparing logs it
would be relatively easy to figure out what is wrong.

I guess that in OP's setup something had changed readtable before, but
this does not happen anymore. And obviously many people would face
similar problems.

This tracing tool should help a lot.

I believe this tool should be supplied by asdf team. Even I begin to
be more positive towards efforts of ASDF team to clean up all the mess
that was in ASDF initially, but obviously society is not quite happy
with breaking changes, so some small tool with a good manual would
make life easier.

Printing readtable before loading, I think, requires just a line or
two. Dumping log of operations might be one (trace) call, so that's
trivial for the person who knows how ASDF is organized. Writing a
small two-paragraph addition to manual would relief a lot of pain and
stress for all.

WBR, Budden

Post by Robert Goldman
I'm with Faré on this one. I don't see evidence that this change is
because ASDF is doing something bad. I believe it's consistent with the
hypothesis that there was some imperfectly-controlled aspect of building
that is done differently now (e.g., files loaded in a different order
where both orders are compatible with the constraints in the system
definitions).
This doesn't even look like an ASDF error to me -- it looks like an
error that occurred while loading a system that ASDF has re-packaged.
So we might be able to help you debug this, but without more evidence,
there's no reason to believe that ASDF has done anything to you: it
looks like the change in ASDF just reveals a pre-existing bug.

[Faré]

Post by 73budden .
Warning about modifying standard readtable is issued by SBCL (at least
in my SBCL 1.3.18), please grep for the message in SBCL sources, and
it seem to be introduced in 1.0.24. I took a look at puri's definition
(maybe old, in my local copy of quicklisp) and it looks like it
actually tries to modify standard readtable.

More precisely, puri modifies the _current_ *readtable*, which is very rude.
If anyone maintains puri (fat luck), he should remove this bug
and instead offer an interface using named-readtables.

The error happens when the current readtable happens to be the
standard readtable, which will happen if someone uses
with-standard-io-syntax.

Looking at ASDF sources, there is one place that uses it, and indeed,
a bug was introduced during the refactoring of ASDF 3.3.0: whereas
earlier variants of ASDF
saved the *readtable* from outside the with-standard-io-syntax to
restore it inside,
the new ASDF fails to do it. Reduced test case: try to
(asdf:load-system "ddop") with the following ddop.asd.

(defsystem "ddop" :defsystem-depends-on ("puri"))

I believe this justifies a 3.3.1 bugfix release indeed.

It also justifies spending more time on the syntax-control branch
supposed to cleanup the readtable issues, and/or getting all software
fixed so it never modifies the current readtable unless it explicitly
makes and sets a new readtable.

Post by 73budden .
It'd be very nice to have instructions on how to trace what happens
while system is being built:that is, which files are compiled, which
are loaded and what is a readtable in the beginning of each load
operation.

You can pass a :verbose t flag to load-system and its friends.

As for tracing what readtable are used, there does not exist
infrastructure for even identifying readtables. You're welcome to
write one and seek assistance in writing
a hook in ASDF (e.g. as a :before method on perform compile-op or load-op,
which would not have helped you much in this case, or as a local
modification to uiop:compile-file*, if you have a local ASDF source
checkout).

Post by 73budden .
Running builds in old and new asdf versions and comparing logs it
would be relatively easy to figure out what is wrong.

Only if given enough information to reproduce, which the complainer
failed to provide. Luckily, I investigated nevertheless.

For those in a hurry for a fix, here is the merge request:
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
The two most common things in the Universe are hydrogen and stupidity.
— Harlan Ellison

[Robert Goldman]

Post by Faré
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85

I will merge this into master, but do not have time to make a new
release, or even test much this week or next.

So this will simply remain in master for now.

[73budden .]

Wow! Good news.

Post by Robert Goldman

Post by Faré
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85

I will merge this into master, but do not have time to make a new
release, or even test much this week or next.
So this will simply remain in master for now.

[Robert Goldman]

Post by Robert Goldman

Post by Faré
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85

I will merge this into master, but do not have time to make a new
release, or even test much this week or next.
So this will simply remain in master for now.

This patch is now available in ASDF 3.3.0.1

[Faré]

Post by Robert Goldman

I will merge this into master, but do not have time to make a new release,
or even test much this week or next.
So this will simply remain in master for now.

This patch is now available in ASDF 3.3.0.1

I also pushed a regression test directly to master.

I will also try to salvage the most important stuff to salvage from
the syntax-control branch, namely some *shared-readtable* and
*shared-print-pprint-dispatch* tables in asdf (or uiop?) to restore
around the build (e.g. in asdf:operate) so it is well-defined what
readtable is used when building with asdf (i.e. the readtable active
when asdf was initially loaded, usually your Lisp's initial
readtable).

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
[T]he “evolution stops at the neck” reflex. They're willing to accept that
everything else about humans has evolved through evolution but the thing
that is most important in explaining your personhood, namely your mind,
somehow evolution doesn’t apply to it. — Gad Saad

[73budden .]

Warning about modifying standard readtable is issued by SBCL (at least
in my SBCL 1.3.18), please grep for the message in SBCL sources, and
it seem to be introduced in 1.0.24. I took a look at puri's definition
(maybe old, in my local copy of quicklisp) and it looks like it
actually tries to modify standard readtable. But here it does not
happen when it is being loaded via my patched asdf 3.1.6 and my own
IDE with forked SLIME with many systems pre-loaded. I don't know why
and have no time to investigate further. Obviously, my insights would
be of limited use because my environment is very different from that
of OP.

Post by Robert Goldman
So we might be able to help you debug this

It'd be very nice to have instructions on how to trace what happens
while system is being built:that is, which files are compiled, which
are loaded and what is a readtable in the beginning of each load
operation.

Running builds in old and new asdf versions and comparing logs it
would be relatively easy to figure out what is wrong.

I guess that in OP's setup something had changed readtable before, but
this does not happen anymore. And obviously many people would face
similar problems.

This tracing tool should help a lot.

I believe this tool should be supplied by asdf team. Even I begin to
be more positive towards efforts of ASDF team to clean up all the mess
that was in ASDF initially, but obviously society is not quite happy
with breaking changes, so some small tool with a good manual would
make life easier.

Printing readtable before loading, I think, requires just a line or
two. Dumping log of operations might be one (trace) call, so that's
trivial for the person who knows how ASDF is organized. Writing a
small two-paragraph addition to manual would relief a lot of pain and
stress for all.

WBR, Budden

Post by Robert Goldman
I'm with Faré on this one. I don't see evidence that this change is
because ASDF is doing something bad. I believe it's consistent with the
hypothesis that there was some imperfectly-controlled aspect of building
that is done differently now (e.g., files loaded in a different order
where both orders are compatible with the constraints in the system
definitions).
This doesn't even look like an ASDF error to me -- it looks like an
error that occurred while loading a system that ASDF has re-packaged.
So we might be able to help you debug this, but without more evidence,
there's no reason to believe that ASDF has done anything to you: it
looks like the change in ASDF just reveals a pre-existing bug.

[Robert Goldman]

All signs point at puri being the culprit. I am absolutely not
disagreeing
with that. I provided the offending code in the bug report myself. The
question is: what to do about it? The project is dead. The home page
is
gone. Here is what I will do: I'll see if I can get drakma to steer
clear
of it. They will undoubtedly be interested in not having their fine
code
dragged down by a dead library.
The only other option is to fix puri. I don't even know how to begin
doing
that with a dead project. Where did Kevin Rosenberg disappear off to?

Subject: Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
Date: Thu, 12 Oct 2017 20:09:44 +0300
Warning about modifying standard readtable is issued by SBCL (at least
in my SBCL 1.3.18), please grep for the message in SBCL sources, and
it seem to be introduced in 1.0.24. I took a look at puri's
definition
(maybe old, in my local copy of quicklisp) and it looks like it
actually tries to modify standard readtable. But here it does not
happen when it is being loaded via my patched asdf 3.1.6 and my own
IDE with forked SLIME with many systems pre-loaded. I don't know why
and have no time to investigate further. Obviously, my insights would
be of limited use because my environment is very different from that
of OP.

Post by Robert Goldman
So we might be able to help you debug this

It'd be very nice to have instructions on how to trace what happens
while system is being built:that is, which files are compiled, which
are loaded and what is a readtable in the beginning of each load
operation.
Running builds in old and new asdf versions and comparing logs it
would be relatively easy to figure out what is wrong.
I guess that in OP's setup something had changed readtable before, but
this does not happen anymore. And obviously many people would face
similar problems.
This tracing tool should help a lot.
I believe this tool should be supplied by asdf team. Even I begin to
be more positive towards efforts of ASDF team to clean up all the mess
that was in ASDF initially, but obviously society is not quite happy
with breaking changes, so some small tool with a good manual would
make life easier.
Printing readtable before loading, I think, requires just a line or
two. Dumping log of operations might be one (trace) call, so that's
trivial for the person who knows how ASDF is organized. Writing a
small two-paragraph addition to manual would relief a lot of pain and
stress for all.
WBR, Budden

Post by Robert Goldman
I'm with Faré on this one. I don't see evidence that this change
is
because ASDF is doing something bad. I believe it's consistent with the
hypothesis that there was some imperfectly-controlled aspect of building
that is done differently now (e.g., files loaded in a different order
where both orders are compatible with the constraints in the system
definitions).
This doesn't even look like an ASDF error to me -- it looks like an
error that occurred while loading a system that ASDF has
re-packaged.
So we might be able to help you debug this, but without more evidence,
there's no reason to believe that ASDF has done anything to you: it
looks like the change in ASDF just reveals a pre-existing bug.

--
Carlos Konstanski
MTS IV Cslt
Verizon Cloud Platform
Cell: 15126218301

That may be, but it was unfair to get angry at the ASDF maintainers
about this. This is just a pre-existing error that was *manifested*
because of a change in ASDF. It's not our fault that this error
appeared, it's not our fault that the puri library is no longer
maintained, and we can't be expected to avoid releasing improved
versions of ASDF because there exists buggy, unmaintained code in
Quicklisp.

An apology from you would be appropriate at this point.

[Stelian Ionescu]

That may be, but it was unfair to get angry at the ASDF maintainers 
about this.  This is just a pre-existing error that was *manifested* 
because of a change in ASDF.  It's not our fault that this error 
appeared, it's not our fault that the puri library is no longer 
maintained, and we can't be expected to avoid releasing improved 
versions of ASDF because there exists buggy, unmaintained code in 
Quicklisp.

No it's not your fault, but I think it would be a very sensitive idea
to avoid annoying ASDF users, simply for practical reasons, because
you'll find yourself with people who fork or refuse to update ASDF.

Something more useful would be to:
 * introduce the notion of ASDF compilation options, as a set of key-
value pairs which control different compilation modes or effects
 * make the new strictness modes, like preserving the readtable, depend
on those toggles, but upon introduction the default should be perfect
backwards-compatibility, even if that is something you consider broken
 * blog about the fancy new way toggle and explain why it's better to
use it than not
 * let the libraries' users nag the developers to change the code to be
compatible with the new strictness checks
 * wait a couple of years (at least) until you see that most of
Quicklisp libraries have been ported to the new way of doing things and
if that happens maybe consider turning it on by default. In that case,
announce it publicly
 * if adoption didn't happen, keep it disabled happily knowing that you
can always turn it on in your company's internal projects.

This is more or less how we do things at work. It has a certain amount
of overhead but it gains you good will from the community; on the other
hand enabling things by default, and on a short notice, only makes you
seem like you're imposing your preference on everybody else just for
the sake of it. I think it's better to let things sink in and allow the
users of ASDF to come to a consensus, although that's a slow process.

[Faré]

Post by Stelian Ionescu

Post by Robert Goldman
That may be, but it was unfair to get angry at the ASDF maintainers
about this. This is just a pre-existing error that was *manifested*
because of a change in ASDF. It's not our fault that this error
appeared, it's not our fault that the puri library is no longer
maintained, and we can't be expected to avoid releasing improved
versions of ASDF because there exists buggy, unmaintained code in
Quicklisp.

No it's not your fault, but I think it would be a very sensitive idea
to avoid annoying ASDF users, simply for practical reasons, because
you'll find yourself with people who fork or refuse to update ASDF.

Well, in this case, it was a combination of things that are not our
fault, and things that are. So my apologies for the things that were
my fault.

Post by Stelian Ionescu
* introduce the notion of ASDF compilation options, as a set of key-
value pairs which control different compilation modes or effects

ASDF already kind of has that, as keyword arguments to operate. They
was a bit of a cleanup in ASDF 3.3.0 already: the arguments are not
passed blindly into non-sensical arguments to the operation class
anymore, and are now well-defined. Thus they can now be used the way
you propose.

Post by Stelian Ionescu
* make the new strictness modes, like preserving the readtable, depend
on those toggles, but upon introduction the default should be perfect
backwards-compatibility, even if that is something you consider broken

Agreed. Any new strictness should be added cautiously and slowly,
disabled by default or adding only a (style-)warning. In this case,
the strictness was added by accident (because the forms that prevented
the strictness were victim of a refactoring I botched while adding a
different feature, and the strictness was only added in a corner case
for which I had no test, but just added one now).

Post by Stelian Ionescu
* blog about the fancy new way toggle and explain why it's better to
use it than not

Will do, if I keep hacking on it.

Post by Stelian Ionescu
* let the libraries' users nag the developers to change the code to be
compatible with the new strictness checks

Maybe. Historically, the main problem has been unmaintained libraries.

Post by Stelian Ionescu
* wait a couple of years (at least) until you see that most of
Quicklisp libraries have been ported to the new way of doing things and
if that happens maybe consider turning it on by default. In that case,
announce it publicly

Yes, we've typically waited for a year or two before we added any new
strictness (e.g. everything is now UTF-8 by default).

Post by Stelian Ionescu
* if adoption didn't happen, keep it disabled happily knowing that you
can always turn it on in your company's internal projects.

This has happened in the past, e.g. for properly catching deferred
warnings. Most users never bothered to do it, maintainers never
bothered to fix their libraries, etc. Maybe with more outreach they
would, but I'm kind of burned out with CL right now.

Post by Stelian Ionescu
This is more or less how we do things at work. It has a certain amount
of overhead but it gains you good will from the community; on the other
hand enabling things by default, and on a short notice, only makes you
seem like you're imposing your preference on everybody else just for
the sake of it. I think it's better to let things sink in and allow the
users of ASDF to come to a consensus, although that's a slow process.

I've tried very hard not to do that this time. It was a bug. (Unlike
four to five years ago, when I learned the hard way not to push too
hard the deferred warning support or the syntax-control branch; the
latter just came back and bit me.)

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Amateur bureaucrats are often even worse than professional bureaucrats.
— John McCarthy

[Robert Goldman]

Post by 73budden .
This tracing tool should help a lot.
I believe this tool should be supplied by asdf team. Even I begin to
be more positive towards efforts of ASDF team to clean up all the mess
that was in ASDF initially, but obviously society is not quite happy
with breaking changes, so some small tool with a good manual would
make life easier.

This will probably not happen for a while -- my work on other aspects,
and just staying on top of bugs and testing takes most of the available
time.

But I think I can provide pieces of a recipe for this kind of debugging
and if someone out there could give feedback, I will see to it that the
recipe gets into the manual.

I think if you want to see the plan that ASDF produces to perform a
requested operation, you should use something like:
```
(setf *plan* (asdf/plan:make-plan nil (make-operation 'load-op)
(find-system "sysname")))
```

Then, to figure out what's happening, I would suggest
```
(trace asdf:operation-done-p)
```
...to see if ASDF is wrong about whether or not it needs to do some
operations.

Then I would try something like tracing `PERFORM`.

I'd have to think a little about what to do if `MAKE-PLAN` gives you a
plan you don't expect.

cheers,
r

Post by 73budden .
Printing readtable before loading, I think, requires just a line or
two. Dumping log of operations might be one (trace) call, so that's
trivial for the person who knows how ASDF is organized. Writing a
small two-paragraph addition to manual would relief a lot of pain and
stress for all.