So I have been trying to bring my latest project into compliance with the no mixing of OS calls/MUI GUI and ixemul. Now that cURL and SSL have been updated and included in the SDK that has been helpful. Here is my problem, I am getting these errors when linking to the libnix version of libxml2-2.6.22:

****************
/Work/Development/gg/bin/../lib/gcc-lib/ppc-morphos/9.3.0/../../../../ppc-morphos/bin/ld: /usr/lib/libnix/libxml2.a(encoding.o): in function `xmlIconvWrapper':
/Varasto/Lähdekoodit/libxml2-2.6.22/encoding.c:1714: undefined reference to `errno'
/Work/Development/gg/bin/../lib/gcc-lib/ppc-morphos/9.3.0/../../../../ppc-morphos/bin/ld: /Varasto/Lähdekoodit/libxml2-2.6.22/encoding.c:1714: undefined reference to `errno'
/Work/Development/gg/bin/../lib/gcc-lib/ppc-morphos/9.3.0/../../../../ppc-morphos/bin/ld: /usr/lib/libnix/libxml2.a(xmlIO.o): in function `__xmlIOErr':
/Varasto/Lähdekoodit/libxml2-2.6.22/xmlIO.c:219: undefined reference to `errno'
/Work/Development/gg/bin/../lib/gcc-lib/ppc-morphos/9.3.0/../../../../ppc-morphos/bin/ld: /Varasto/Lähdekoodit/libxml2-2.6.22/xmlIO.c:219: undefined reference to `errno'
/Work/Development/gg/bin/../lib/gcc-lib/ppc-morphos/9.3.0/../../../../ppc-morphos/bin/ld: /usr/lib/libnix/libxml2.a(nanohttp.o): in function `socket_errno':
/Varasto/Lähdekoodit/libxml2-2.6.22/nanohttp.c:183: undefined reference to `errno'
/Work/Development/gg/bin/../lib/gcc-lib/ppc-morphos/9.3.0/../../../../ppc-morphos/bin/ld: /usr/lib/libnix/libxml2.a(nanohttp.o):/Varasto/Lähdekoodit/libxml2-2.6.22/nanohttp.c:183: more undefined references to `errno' follow
collect2: error: ld returned 1 exit status
make: *** [Makefile:12: Setlist-FM-dl] Error 1
*******************

I know it's an old version of libXML2, but it is the only one readily available that I was able to build libxslt against. I noticed in previous news items that these components have been updated, the 3.0 and 3.12 releases mention them:

"Updated to libXML2 2.7.8 and libXSLT 1.1.26"

I have tried multiple things, the order of the link libs, adding <errno.h>, etc. but nothing seems to work even though errno is defined in multiple libs...Here is a portion of my makefile:

CC = ppc-morphos-gcc-9 -Wall
CFLAGS = `xml2-config --cflags`
LIBS = -lcurl -lnghttp2 -ldl -lssl -lcrypto -lpthread -lxslt -noixemul `xml2-config --libs`

Any help would be appreciated...particularly if the newer versions of the libnix versions of libXML2 and libXSLT were made available.

Need to rebuild this lib with last SDK (i think to fix this err)

i try to building libxml2 (2.9.10), i think it's easy :
You can test :

https://www.morphos-storage.net/upload/test/libxml2-2.9.10.lha

You can find all src and lib in .libs and includes

I compile it with that :
configure CFLAGS=-noixemul LDFLAGS=-noixemul --with-threads=no --enable-shared=no --with-ftp=no --with-http=no



[ Edité par beworld 03.01.2021 - 12:31 ]

Any linklib which used old SDK (and global errno) needs to be rebuilt.

Correct. These errors arise from the fact that errno (among with some other variables) are now per-thread, and are no longer regular variables. Rebuilding any link libraries should fix these issues, indeed.

This change is due to need to have per-thread errno for multi-threaded applications. Before this change all threads shared a single variable and would corrupt the errno of each other.

For the interested: The actual implementation uses the new exec TLS functionality, with fallback of using old shared variable if ran on older systems.

For the even more interested: Look into /gg/includestd/gentls.h for an example on how to use the exec TLS to implement per-thread variables. You can use this header like this:

Code:

Now, to access per-thread foo you can just use (*__local_foo()) instead of foo.

libnix itself uses this header to implement errno (and the other per-thread variables). errno itself is a macro now
Code:
... where __errno_location is pretty 100% the same as __local_foo in the above example.

Of course you're not limited to using this header by any means, it's just provided to help implement things quickly and without much extra effort. Nothing prevents you from using the exec functions directly yourself of course.

[ Edited by Piru 03.01.2021 - 14:20 ]

Oh
Thanks for the explanation

Quote:

Btw, the example code in TLSCallDestructors() autodoc using SIGF_SINGLE is not really safe.

Random wild example: if current or future implementation of CreateNewProc() did some debug output after AddTask() and debug output for some reasons happens to need to do some semaphore locking then the created task doing (Signal(parent, SIGF_SINGLE)) could erranouesly wake up the CreateNewProc() which by bad luck is still blocked in ObtainSemaphore() call.

Thanks BeWorld for the more recent libXML2, I'll take a look at at it.

Thanks Piru for this excellent explanation. This helps me understand this a lot!

Quote:
Such change to CreateNewProc() would prove rather dangerous in my opinion. However, I will amend this autodoc for the next SDK to avoid giving even potentially dangerous example.

Quote:

I'm not sure I understand what you mean with change. A debug output is not really a change.

Maybe my example is bad. Better example: I have seen in (old) sources that you have a FreeTagItems() after the AddTask() in CreateNewProc(). Sometime in future someone may optimize utility tag allocation functions to use semaphore protected memory pool, instead of AllocVec()/FreeVec() -> same problem -> created task may signal he's done with init and SIGF_SINGLE erranously wakes up parent task still inside createnewproc() waiting for memory pool semaphore.

Quote:
Yes it's a change, if it changes the (even unspecified) behaviour of the function that existing applications rely on. Numerous applications depend on CreateNewProc() (with the specific set of tags being passed to it) not breaking Forbid(). Such change would break these applications in a random way. This includes far wider range of applications than ones abusing SIGF_SINGLE.

Quote:
It's still a bad example, because:

1) It wouldn't be an optimization by any measure, but rather an API breaking change. The utility.library tag manipulation function memory can't be allocated from per task pool since no-one has specified it illegal to pass these pointers between tasks. And yes, different tasks allocate and free the taglists, so this would be an unsafe change.

2) If the taglists are global, so semaphore protected memory offers zero optimization over AllocVec()/FreeVec() anyway. This is the case with MorphOS at least, where regular allocation functions are marginally faster than memory pools. Memory pools are handy since they track allocations, but that's pretty much it. They're still a bit slower than regular memory functions.

3) Same thing applies as before, many things rely on this utility.library tag interface not breaking Forbid() so any use of semaphores (or other functionality that might indirectly Wait()) is out.

The example has been changed to avoid the use of SIGF_SINGLE. This doesn't mean that MorphOS CreateNewProc() might suddenly start using semaphores or otherwise break Forbid(). This won't happen for the above mentioned reasons.


PS. I use term "API" loosely here. I probably should use "ABI", or even something more restrictive since many of these things are implementation specific dependencies. Yet, the fact is that existing apps do rely on these undocumented behaviours.

[ Edited by Piru 04.01.2021 - 15:09 ]

1) What I meant was a normal global memory pool, not per task. Sem protected either automatically (MEMF_SEM_PROTECTED) or manually. And the optimization being about less memory fragmentation at least with standard old style Amiga memory allocation which MorphOS still allows to use with some boot option to override the otherwise better newer TLSF thing, right?

Quote:
Semaphore protected memory pools have no meaning under MorphOS, except the automatic alloc tracking and easy deletion of all pool memory. Other than that it's just slower than regular memory allocations. See below.

Quote:
I'd argue that the reason exec memory pools were added in the first place was due to the standard AmigaOS "First Fit" allocator being so awfully slow when fragmentation got high. They weren't added to try fix the cause but the mitigate the effect. Of course the official story was that they were added to fix the fragmentation problem (but they really don't do that, not really).

Hence in most cases switching anything to use memory pools from regular memory allocations is just a mitigation to the horrible AmigaOS "first fit" allocator. In MorphOS context it makes no sense (unless if you also want the automatic tracking for some reason).

In fact, memory pools as they were originally implemented would have actually slowed down the system massively when I implemented TLSF allocator. So I just ripped the old implementation out and made it pretty much just a AllocMem() + AddHead() for AllocPooled() and have a list in the pool header for the allocations. FreePooled() is just Remove() + FreeMem(). Allocate() and Deallocate() are never used. The Allocate() and Deallocate() functions are there and fully functional for anyone insane enough to call them, of course, but internally MorphOS doesn't use them at all.

In fact, in the MorphOS memory pool implementation MEMF_SEM_PROTECTED only protects the AddHead() and Remove() calls, which are super fast in themselves, anyway. So in practice under MorphOS there's actually small performance penalty from using MEMF_SEM_PROTECTED memory pools. It is a small one and something you might want to pay to get the automatic tracking and free at DeletePool().

Quote:
No-one does that. The only reason you'd ever have for using them is going away, too, with built-in wipeout kind of memory corruption tracking coming to MorphOS 3.16 (permmemtrack boot option, which earlier only worked with the old-style memory allocator).

Anyway, long story short: MorphOS is not going to swich to MEMF_SEM_PROTECTED memory pools internally. It doesn't make any sense.

This is getting a bit off-topic for "errno", I fear.

[ Edited by Piru 04.01.2021 - 17:18 ]