I say it's difficult. Maybe someone has a better idea...?

I know why it doesn't work in HXA. There is a single "fetch next input line" function. Disregarding details, the core logic is:



Macro expansions take precedence over file reads. So when a macro is expanded, the assembler "naturally" suspends input from the current input file and processes macro lines instead.

What if you want to nest macro expansions? That's fine. There's some behind-the-scenes stacking, but as far as the "fetch next input line" function is concerned, all that matters is a macro is active.

What if you want to nest file inclusions? That's fine. There's some behind-the-scenes stacking, but as far the "fetch next input line" function is concerned, all that matters is the file is not empty. When it does become empty, then there's some behind-the-scenes unstacking to restore the calling file (and if there isn't one, then this pass is done).

Incidentally, you'd want to keep track of what the current input line number is in each stacked input file, so it can be restored properly as the file inclusion stack unwinds. You'd want to do that so you can report any errors with the proper line numbers for any input file, of course.

What if you want to expand a macro within an included file? That's fine. Nothing special needs to be done that isn't already done. The macro expansion suspends further reading of the included file until it's done expanding (however deep it nests).

Okay, here's the rub: what if you want to include a file within a macro expansion? Hmmm...there's no real problem with switching file input to the include file. The file-stacking logic still works just fine. However, the next time the "fetch next input line" function is called, as far as it's concerned a macro is still active. So the next input line will come from the macro expansion, not the include file.

When will input start coming from the include file? Only when the macro expansion stops. So you get the effect of "delayed" reading of the include file.

Is there a work around? Sure. If the include file instruction is the LAST line of the macro (which is itself not a nested macro), then there's no more macro to expand. So the next line WILL come from the include file. Nothing's really changed as far as how things work or why they do, but it appears to work "as expected".

So the question: should this be considered a "bug"? Is there a simple design I've overlooked that would "naturally" suspend processing of a macro expansion in favor of an include file without a lot of fuss?

I haven't given it a whole lot of thought, but if it was possible to make a macro expansion "look like" a file it might not be so bad. I can even see actually writing out a complete macro expansion to a temporary file and then switching input to that file. Is the capability so important that it would be worth going to those lengths?

In Dev65 I use a stack of objects to track where I am currently getting source from. All the objects on the stack derive from abstract class that defines an API for extracting lines of text. This class is specialized into classes for files, macros and repeats. The objects representing macro sources are built when you define a macro and kept indexed by name in a hash table for the pass. The others are transiently built and destroyed as needed while processing the source.

So you can use any kind of source inside another type of source if you needed.

_________________
Andrew Jacobs
6502 & PIC Stuff - http://www.obelisk.me.uk/
Cross-Platform 6502/65C02/65816 Macro Assembler - http://www.obelisk.me.uk/dev65/
Open Source Projects - https://github.com/andrew-jacobs

I expect it would be related to how the list file is created.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

I am having trouble thinking in a rigorous way, but perhaps someone else will.

First of all, there is a precedence issue. Macros, then file inclusion. Is there a difference? A file inclusion is a kind of macro; a bunch of text is replacing the include statement.

There also exists a 'macro dictionary' that is manipulated throughout the process. I like to think in terms of Forth dictionaries and parsing, but assemblers allow forward referencing and do multiple passes. I am confused by this - what happens if macros are redefined at the end? The earlier versions are never executed. So macro redefinition must be handled at expansion-time, and forward references must not be allowed for macros.

Here is the logical to approach expansion. Consider that there is a 'source pointer' and the 'destination pointer'. Both start in the same spot in the beginning. As macro definitions are encountered, they are placed into the dictionary. As macro invocations are encountered, they are expanded into the destination pointer. Since the expansion happens 'into the future', the expanded text is re-scanned, allowing for nested macro invocations or definitions.

For error handling, the assembler must keep track of all the line numbers in a stacked form through macro expansions.

_________________
In theory, there is no difference between theory and practice. In practice, there is. ...Jan van de Snepscheut

The whole point of a macro is that the parameters you call it with change the code it generates. A file inclusion typically just replicates the contents of the file unless you use equates and conditional assembly.

An assembler normally requires macros to be defined before they are used.

Yes, a macro assembler needs some kind of stack of text sources. Macros stored in memory probably also need to know where they were defined so errors can be correctly reported. In my assembler every line of text read from a source also has a location that indicates the file and line it came.

Get over your fear of Java and try it. You only need the JRE and if you have a browser installed you may already have it.

_________________
Andrew Jacobs
6502 & PIC Stuff - http://www.obelisk.me.uk/
Cross-Platform 6502/65C02/65816 Macro Assembler - http://www.obelisk.me.uk/dev65/
Open Source Projects - https://github.com/andrew-jacobs

True enough, but you can have macros with no parameters. I don't think treating them differently or prioritizing one over the other is necessary, anyway

Oh, it's not a fear - I was really into Java when it first came out. Having used it extensively I just became annoyed with it and can't find any reason to use it over C/C++ and generally try to stay away from it.

_________________
In theory, there is no difference between theory and practice. In practice, there is. ...Jan van de Snepscheut

At a high level, there's could be little difference between an include and a macro. You have a line of code in the source that gets expanded.

Operationally, they're different if for no other reason than with error reporting, but even then they could be remarkably similar.

As for includes define macros or macros having includes, the input processor stack can manage that, and obviously things like includes need to have some grasp of the global context so that you can enforce that a file isn't being included by itself (which would be bad). You may have a similar restriction with recursive macro expansion. But that's easy enough to detect.

Regarding macro redefinition, most assemblers aren't complicated enough to have multiple scopes, save perhaps for local label resolution. So, you can simply not allow macro redefinition. Even if you allow them to be redefined, there's no reason that redefinition of an executing macro has to take immediate effect. You simply capture the macro meta data during expansion, then it will be released and freed after the macro execution.

Not a real big deal.

I don't think macro redefinition should not be scoped anyway, it's a global affair. It really should not be a problem to redefine macros, including recursive definitions (definitions that include the current definition with other things added).

Even recursive file includes are no different. For instance a file may have large conditional assembly blocks, and depending on some state variables assemble completely differently in different inclusions. So it does not have to form an infinite loop, and the assembler is not smart enough to figure it out to cause an error. It would be crude (but effective) to error check for filename repetition.

I still think that macros are macros, file-based or text based, and can be handled with the same simple engine.

EDIT: And if the 'include' is in the middle of a macro definition it is like any other text. After the macro is expanded the include will be encountered and handled...

It seems that you can either let things naturally and sensibly fall into place or create arbitrary rules for no good reason (it seems that everyone does the latter).

_________________
In theory, there is no difference between theory and practice. In practice, there is. ...Jan van de Snepscheut