In a recent email thread on the ACCU-general mailing list, Adrian Fagg came out with the immortal statement:
"You might think that calling direct to C++ across those boundaries is a triumph of technology, I think it's a siren luring you onto the rocks."
This is so cool, I'm almost speechless. (Yes, my friends, I could never actually be speechless. Perhaps I should say "This is so cool, I'm experiencing a temporary diminution of my garrullous and overweening eloquence"? Perhaps not.)
Anyhoo, this would have a been a great quote to lead into part 2 of Imperfect C++, which covers a number of issues regarding C++'s non-standard, tricky, and limiting attempts at ABIs.
Sunday, June 20, 2010
Friday, June 11, 2010
Imperfect C++ Inventions: dimensionof
Characterisation
dimensionof() is a macro, also known as STLSOFT_NUM_ELEMENTS(), for compile-time array size evaluation.
Purpose
Its purpose is to calculate, at compile time, the dimension of an array, whilst rejecting, as a compile-error, application of the macro to a pointer or an instance of a user-defined type that has a subscript operator.
History
When I started programming C and C++ in the early 1990s, one of the first things I did was create a NUM_ELEMENTS() macro, defined as:
#define NUM_ELEMENTS(x) (sizeof(x)/sizeof((x)[0]))
Some years later, after reading Peter van der Linden's excellent Deep C Secrets, I realised I could reverse the subscript operator, and instead define it as:
#define NUM_ELEMENTS(x) (sizeof(x)/sizeof(0[(x)]))
Then, as part of the STLSoft project, I worked out a method of applying function templates taking references to arrays of a deducible dimension, in the form of the macro STLSOFT_NUM_ELEMENTS(). The details of the technique are discussed in chapter 14 of Imperfect C++, and the implementation shown in the online documentation.
In the latest alpha release of STLSoft 1.10, the header file stlsoft/util/dimensionof.h introduces the dimensionof() macro, as a syntactic alias for STLSOFT_NUM_ELEMENTS(). The reason it's in a separate header is because, unlike STLSOFT_NUM_ELEMENTS(), it does not have an unambiguous name.
Dependencies
It has no dependencies.
Uses
Throughout the STLSoft libraries, and also used widely in dependent libraries such as b64, FastFormat, Pantheios, recls and VOLE.
Further Reading
dimensionof() is a macro, also known as STLSOFT_NUM_ELEMENTS(), for compile-time array size evaluation.
Purpose
Its purpose is to calculate, at compile time, the dimension of an array, whilst rejecting, as a compile-error, application of the macro to a pointer or an instance of a user-defined type that has a subscript operator.
int ar[1];
int* pi = &ar[0];
std::vector vi;
STLSOFT_NUM_ELEMENTS(ar); // Ok
STLSOFT_NUM_ELEMENTS(pi); // compile error
STLSOFT_NUM_ELEMENTS(vi); // compile error
History
When I started programming C and C++ in the early 1990s, one of the first things I did was create a NUM_ELEMENTS() macro, defined as:
#define NUM_ELEMENTS(x) (sizeof(x)/sizeof((x)[0]))
Some years later, after reading Peter van der Linden's excellent Deep C Secrets, I realised I could reverse the subscript operator, and instead define it as:
#define NUM_ELEMENTS(x) (sizeof(x)/sizeof(0[(x)]))
Then, as part of the STLSoft project, I worked out a method of applying function templates taking references to arrays of a deducible dimension, in the form of the macro STLSOFT_NUM_ELEMENTS(). The details of the technique are discussed in chapter 14 of Imperfect C++, and the implementation shown in the online documentation.
In the latest alpha release of STLSoft 1.10, the header file stlsoft/util/dimensionof.h introduces the dimensionof() macro, as a syntactic alias for STLSOFT_NUM_ELEMENTS(). The reason it's in a separate header is because, unlike STLSOFT_NUM_ELEMENTS(), it does not have an unambiguous name.
Dependencies
It has no dependencies.
Uses
Throughout the STLSoft libraries, and also used widely in dependent libraries such as b64, FastFormat, Pantheios, recls and VOLE.
Further Reading
- The most comprehensive explanation is to be found in chapter 14 of Imperfect C++.
Friday, December 4, 2009
Imperfect C++ Inventions #1: auto_buffer
Characterisation
auto_buffer is a C++ class template.
Purpose
It's purpose is to avoid memory allocation costs in cases where the sizes of most, but not all, allocations for a given operation are known to occur below a certain threshold. It is intended to be used as a variable in automatic scope (i.e. a stack variable). This is very common in cases such as allocating space for a file-system path, where most fall within, say, 100 characters, but rare cases may extend to several thousand characters: using a stack buffer of 100 would lead to overruns, using a stack buffer of 10,000 would, even if above the maximum value, lead to undesirable stack-related performance effects.
It is implemented in terms of a standard allocator-conformant type, determined via the specialisation of the auto_buffer class template, defaulted to a specialisation of std::allocator. By design, it can be specialized only by POD types, since one of the . As noted in section 16.2.1 of Extended STL, volume 1, despite, for convenience, providing many methods found on standard containers, auto_buffer is not a container, and should ever be mistaken for such.
As extensive performance studies (and a lot of real-world use) have demonstrated, auto_buffer is effectively a way of having (almost) all the performance of stack-based memory allocation while having the flexibility of heap-based memory allocation. Furthermore, it is portable and has no drawbacks as regards its intended purpose; all other options, such as alloca() and Variable-Length Arrays, have unacceptable behaviour and/or portability and/or language-specificity, as described in detail in chapter 32 of Imperfect C++.
History
auto_buffer had a very prosaic start in life. Long ago, in a programming fashion far, far away, I was implementing a suite of utility functions providing a multibyte string conversion layer over an API that was implemented in terms of wide strings. The first implementation used the heap to allocate a block of memory in which to write the converted form, before passing on to the underlying API. Understandably, this rankled, particularly when most of the converted strings fell well within a buffer size that one might comfortably use on the stack at the time (early 1990s).
The second implementation of these utilities consequently declared a local (stack) buffer, which was used if it could accommodate the given requirement, otherwise the heap was used as before. Later, once enough compilers caught up with the requirements of the type, these implementations were replaced by LocalBuffer, the Synesis Software class template that later evolved into STLSoft's auto_buffer.
Implementations
Implementations of auto_buffer are found for STLSoft and Boost:
It has dependencies only on standard facilities.
Uses
Within STLSoft, auto_buffer is used within many components, including ones for character encoding and conversion, process enumeration, Windows registry, ACE communications component adaptors, formatting functions, file-system processing, to name but a few.
In other open-source libraries it is used to enhance performance at the function-scale, particularly in the high-performance formatting and diagnostic logging libraries, FastFormat and Pantheios.
Being a low-level component - in particular, being (by design) only specializable for POD types - it tends not to be used too often in application code. Nonetheless, in several commercial applications in recent years, I have found a need for the component, to assist in safely optimising performance bottlenecks where the majority of memory allocations fall within a known bound.
Further Reading
auto_buffer is a C++ class template.
Purpose
It's purpose is to avoid memory allocation costs in cases where the sizes of most, but not all, allocations for a given operation are known to occur below a certain threshold. It is intended to be used as a variable in automatic scope (i.e. a stack variable). This is very common in cases such as allocating space for a file-system path, where most fall within, say, 100 characters, but rare cases may extend to several thousand characters: using a stack buffer of 100 would lead to overruns, using a stack buffer of 10,000 would, even if above the maximum value, lead to undesirable stack-related performance effects.
It is implemented in terms of a standard allocator-conformant type, determined via the specialisation of the auto_buffer class template, defaulted to a specialisation of std::allocator. By design, it can be specialized only by POD types, since one of the . As noted in section 16.2.1 of Extended STL, volume 1, despite, for convenience, providing many methods found on standard containers, auto_buffer is not a container, and should ever be mistaken for such.
As extensive performance studies (and a lot of real-world use) have demonstrated, auto_buffer is effectively a way of having (almost) all the performance of stack-based memory allocation while having the flexibility of heap-based memory allocation. Furthermore, it is portable and has no drawbacks as regards its intended purpose; all other options, such as alloca() and Variable-Length Arrays, have unacceptable behaviour and/or portability and/or language-specificity, as described in detail in chapter 32 of Imperfect C++.
History
auto_buffer had a very prosaic start in life. Long ago, in a programming fashion far, far away, I was implementing a suite of utility functions providing a multibyte string conversion layer over an API that was implemented in terms of wide strings. The first implementation used the heap to allocate a block of memory in which to write the converted form, before passing on to the underlying API. Understandably, this rankled, particularly when most of the converted strings fell well within a buffer size that one might comfortably use on the stack at the time (early 1990s).
The second implementation of these utilities consequently declared a local (stack) buffer, which was used if it could accommodate the given requirement, otherwise the heap was used as before. Later, once enough compilers caught up with the requirements of the type, these implementations were replaced by LocalBuffer, the Synesis Software class template that later evolved into STLSoft's auto_buffer.
Implementations
Implementations of auto_buffer are found for STLSoft and Boost:
- The original implementation (corresponding to the discussion in Imperfect C++) is stlsoft::auto_buffer
- Thorsten Ottosen - a valued reviewer of (parts of) Imperfect C++ and (all of) Extended STL, volume 1 - is working on an enhanced version for Boost
It has dependencies only on standard facilities.
Uses
Within STLSoft, auto_buffer is used within many components, including ones for character encoding and conversion, process enumeration, Windows registry, ACE communications component adaptors, formatting functions, file-system processing, to name but a few.
In other open-source libraries it is used to enhance performance at the function-scale, particularly in the high-performance formatting and diagnostic logging libraries, FastFormat and Pantheios.
Being a low-level component - in particular, being (by design) only specializable for POD types - it tends not to be used too often in application code. Nonetheless, in several commercial applications in recent years, I have found a need for the component, to assist in safely optimising performance bottlenecks where the majority of memory allocations fall within a known bound.
Further Reading
- The most comprehensive explanation is to be found in chapter 32 of Imperfect C++.
- There is an auto_buffer-in-5-pages basic grounding in chapter 16 of Extended STL, volume 1.
- The 2003 introductory article in C/C++ Users Journal is still available via DDJ.
Imperfect C++ Inventions: introduction
Recently, I've been asked by a couple of readers to enumerate the inventions and new definitions (as opposed to techniques, or already established technologies) introduced in Imperfect C++. Specifically, what are the (C++) programming components/techniques/technologies that I invented that are documented in the book.
To satisfy this request, I will make a series of posts covering the inventions and new definitions, including:
Note 1: I'm only discussing inventions that are described in Imperfect C++. Things that I've later concocted are/will be discussed elsewhere. For example: the Type Tunnel and Handle::Ref+Wrapper patterns and the Principle of Intersecting Conformance will be discussed in Breaking Up The Monolith; C# Static Extension Methods will be discussed in Imperfect C#; and so on.
Note 2: I'm not claiming in every case that the invention is creditable only to me. Some were joint efforts (e.g. Ranges, with John Torjo); some are my taking an existing technique to new lengths and depths (e.g. Shims); some are my satisfying a known problem in a different way, or providing greater portability over existing forms (e.g. C++ properties). Other than those caveats, however, and the obvious "standing on the shoulders of giants" that is the business of all software designers, these are mine, and any similarity to other inventions, living or dead, is purely coincidental. ;-)
And, of course, there's something of an ulterior motive, in that the full descriptions and design decisions regarding each invention are to be found only in the book, which you may wish to consider getting hold of if you've not done so (particularly now that the reprint will shortly be available).
To satisfy this request, I will make a series of posts covering the inventions and new definitions, including:
- array_proxy
- auto_buffer
- dimensionof
- Empty Derived Optimization (EDO)
- explicit_cast
- fast integer to string conversion
- fast string concatenation
- interface_cast
- multi-dimensional array(s)
- portable vtables
- (C++) properties
- ranges
- shims
- true typedefs
Note 1: I'm only discussing inventions that are described in Imperfect C++. Things that I've later concocted are/will be discussed elsewhere. For example: the Type Tunnel and Handle::Ref+Wrapper patterns and the Principle of Intersecting Conformance will be discussed in Breaking Up The Monolith; C# Static Extension Methods will be discussed in Imperfect C#; and so on.
Note 2: I'm not claiming in every case that the invention is creditable only to me. Some were joint efforts (e.g. Ranges, with John Torjo); some are my taking an existing technique to new lengths and depths (e.g. Shims); some are my satisfying a known problem in a different way, or providing greater portability over existing forms (e.g. C++ properties). Other than those caveats, however, and the obvious "standing on the shoulders of giants" that is the business of all software designers, these are mine, and any similarity to other inventions, living or dead, is purely coincidental. ;-)
And, of course, there's something of an ulterior motive, in that the full descriptions and design decisions regarding each invention are to be found only in the book, which you may wish to consider getting hold of if you've not done so (particularly now that the reprint will shortly be available).
Reprint publication schedule
I've heard from my editor, and the next run of Imperfect C++ went to the printer on 20th November and should be bound on 11th December, so should be available towards the end of the month.
If you are still having issues with availability in the new year, please let me know and I'll contact the publisher for more information.
If you are still having issues with availability in the new year, please let me know and I'll contact the publisher for more information.
Friday, November 20, 2009
Imperfect C++ online content
I thought it'd be useful to list the available online resources for Imperfect C++:
Sample Chapters:
I'll update this list from time to time as I remember more ...
Sample Chapters:
- Chapter 10: Threading - a PDF of the chapter as it appears in the book
- Chapter 25: Fast, Non-intrusive String Concatenation - the article of the same name from C/C++ Users Journal (June 2004)
- Chapter 26: What's Your Address? - the sample chapter in article form, published in The C++ Source (April 2005)
- Chapter 34: Functors and Ranges - a PDF of the chapter as it appears in the book
- Overloading to Enforce C++ Compile-time Constraints, Peter Bannister, Dr Dobb's Journal, May 2005
I'll update this list from time to time as I remember more ...
Thursday, November 19, 2009
Where is "Breaking Up The Monolith"?
Some readers have asked recently (and not so recently, my having taken 3+ years so far) when Breaking Up The Monolith is going to be published?
The short answer is: 2010
The long answer is somewhat more complex, so I'll answer it on Monolith's blog.
The short answer is: 2010
The long answer is somewhat more complex, so I'll answer it on Monolith's blog.
Subscribe to:
Posts (Atom)