src/3rdparty/ptmalloc/malloc.c
changeset 0 1918ee327afb
equal deleted inserted replaced
-1:000000000000 0:1918ee327afb
       
     1 /*
       
     2   $Id: malloc.c,v 1.4 2006/03/30 16:47:29 wg Exp $
       
     3 
       
     4   This version of malloc.c was adapted for ptmalloc3 by Wolfram Gloger
       
     5   <wg@malloc.de>.  Therefore, some of the comments below do not apply
       
     6   for this modified version.  However, it is the intention to keep
       
     7   differences to Doug Lea's original version minimal, hence the
       
     8   comments were mostly left unchanged.
       
     9 
       
    10  -----------------------------------------------------------------------
       
    11 
       
    12   This is a version (aka dlmalloc) of malloc/free/realloc written by
       
    13   Doug Lea and released to the public domain, as explained at
       
    14   http://creativecommons.org/licenses/publicdomain.  Send questions,
       
    15   comments, complaints, performance data, etc to dl@cs.oswego.edu
       
    16 
       
    17 * Version pre-2.8.4 Wed Mar 29 19:46:29 2006    (dl at gee)
       
    18 
       
    19    Note: There may be an updated version of this malloc obtainable at
       
    20            ftp://gee.cs.oswego.edu/pub/misc/malloc.c
       
    21          Check before installing!
       
    22 
       
    23 * Quickstart
       
    24 
       
    25   This library is all in one file to simplify the most common usage:
       
    26   ftp it, compile it (-O3), and link it into another program. All of
       
    27   the compile-time options default to reasonable values for use on
       
    28   most platforms.  You might later want to step through various
       
    29   compile-time and dynamic tuning options.
       
    30 
       
    31   For convenience, an include file for code using this malloc is at:
       
    32      ftp://gee.cs.oswego.edu/pub/misc/malloc-2.8.3.h
       
    33   You don't really need this .h file unless you call functions not
       
    34   defined in your system include files.  The .h file contains only the
       
    35   excerpts from this file needed for using this malloc on ANSI C/C++
       
    36   systems, so long as you haven't changed compile-time options about
       
    37   naming and tuning parameters.  If you do, then you can create your
       
    38   own malloc.h that does include all settings by cutting at the point
       
    39   indicated below. Note that you may already by default be using a C
       
    40   library containing a malloc that is based on some version of this
       
    41   malloc (for example in linux). You might still want to use the one
       
    42   in this file to customize settings or to avoid overheads associated
       
    43   with library versions.
       
    44 
       
    45 * Vital statistics:
       
    46 
       
    47   Supported pointer/size_t representation:       4 or 8 bytes
       
    48        size_t MUST be an unsigned type of the same width as
       
    49        pointers. (If you are using an ancient system that declares
       
    50        size_t as a signed type, or need it to be a different width
       
    51        than pointers, you can use a previous release of this malloc
       
    52        (e.g. 2.7.2) supporting these.)
       
    53 
       
    54   Alignment:                                     8 bytes (default)
       
    55        This suffices for nearly all current machines and C compilers.
       
    56        However, you can define MALLOC_ALIGNMENT to be wider than this
       
    57        if necessary (up to 128bytes), at the expense of using more space.
       
    58 
       
    59   Minimum overhead per allocated chunk:   4 or  8 bytes (if 4byte sizes)
       
    60                                           8 or 16 bytes (if 8byte sizes)
       
    61        Each malloced chunk has a hidden word of overhead holding size
       
    62        and status information, and additional cross-check word
       
    63        if FOOTERS is defined.
       
    64 
       
    65   Minimum allocated size: 4-byte ptrs:  16 bytes    (including overhead)
       
    66                           8-byte ptrs:  32 bytes    (including overhead)
       
    67 
       
    68        Even a request for zero bytes (i.e., malloc(0)) returns a
       
    69        pointer to something of the minimum allocatable size.
       
    70        The maximum overhead wastage (i.e., number of extra bytes
       
    71        allocated than were requested in malloc) is less than or equal
       
    72        to the minimum size, except for requests >= mmap_threshold that
       
    73        are serviced via mmap(), where the worst case wastage is about
       
    74        32 bytes plus the remainder from a system page (the minimal
       
    75        mmap unit); typically 4096 or 8192 bytes.
       
    76 
       
    77   Security: static-safe; optionally more or less
       
    78        The "security" of malloc refers to the ability of malicious
       
    79        code to accentuate the effects of errors (for example, freeing
       
    80        space that is not currently malloc'ed or overwriting past the
       
    81        ends of chunks) in code that calls malloc.  This malloc
       
    82        guarantees not to modify any memory locations below the base of
       
    83        heap, i.e., static variables, even in the presence of usage
       
    84        errors.  The routines additionally detect most improper frees
       
    85        and reallocs.  All this holds as long as the static bookkeeping
       
    86        for malloc itself is not corrupted by some other means.  This
       
    87        is only one aspect of security -- these checks do not, and
       
    88        cannot, detect all possible programming errors.
       
    89 
       
    90        If FOOTERS is defined nonzero, then each allocated chunk
       
    91        carries an additional check word to verify that it was malloced
       
    92        from its space.  These check words are the same within each
       
    93        execution of a program using malloc, but differ across
       
    94        executions, so externally crafted fake chunks cannot be
       
    95        freed. This improves security by rejecting frees/reallocs that
       
    96        could corrupt heap memory, in addition to the checks preventing
       
    97        writes to statics that are always on.  This may further improve
       
    98        security at the expense of time and space overhead.  (Note that
       
    99        FOOTERS may also be worth using with MSPACES.)
       
   100 
       
   101        By default detected errors cause the program to abort (calling
       
   102        "abort()"). You can override this to instead proceed past
       
   103        errors by defining PROCEED_ON_ERROR.  In this case, a bad free
       
   104        has no effect, and a malloc that encounters a bad address
       
   105        caused by user overwrites will ignore the bad address by
       
   106        dropping pointers and indices to all known memory. This may
       
   107        be appropriate for programs that should continue if at all
       
   108        possible in the face of programming errors, although they may
       
   109        run out of memory because dropped memory is never reclaimed.
       
   110 
       
   111        If you don't like either of these options, you can define
       
   112        CORRUPTION_ERROR_ACTION and USAGE_ERROR_ACTION to do anything
       
   113        else. And if if you are sure that your program using malloc has
       
   114        no errors or vulnerabilities, you can define INSECURE to 1,
       
   115        which might (or might not) provide a small performance improvement.
       
   116 
       
   117   Thread-safety: NOT thread-safe unless USE_LOCKS defined
       
   118        When USE_LOCKS is defined, each public call to malloc, free,
       
   119        etc is surrounded with either a pthread mutex or a win32
       
   120        spinlock (depending on WIN32). This is not especially fast, and
       
   121        can be a major bottleneck.  It is designed only to provide
       
   122        minimal protection in concurrent environments, and to provide a
       
   123        basis for extensions.  If you are using malloc in a concurrent
       
   124        program, consider instead using nedmalloc
       
   125        (http://www.nedprod.com/programs/portable/nedmalloc/) or
       
   126        ptmalloc (See http://www.malloc.de), which are derived
       
   127        from versions of this malloc.
       
   128 
       
   129   System requirements: Any combination of MORECORE and/or MMAP/MUNMAP
       
   130        This malloc can use unix sbrk or any emulation (invoked using
       
   131        the CALL_MORECORE macro) and/or mmap/munmap or any emulation
       
   132        (invoked using CALL_MMAP/CALL_MUNMAP) to get and release system
       
   133        memory.  On most unix systems, it tends to work best if both
       
   134        MORECORE and MMAP are enabled.  On Win32, it uses emulations
       
   135        based on VirtualAlloc. It also uses common C library functions
       
   136        like memset.
       
   137 
       
   138   Compliance: I believe it is compliant with the Single Unix Specification
       
   139        (See http://www.unix.org). Also SVID/XPG, ANSI C, and probably
       
   140        others as well.
       
   141 
       
   142 * Overview of algorithms
       
   143 
       
   144   This is not the fastest, most space-conserving, most portable, or
       
   145   most tunable malloc ever written. However it is among the fastest
       
   146   while also being among the most space-conserving, portable and
       
   147   tunable.  Consistent balance across these factors results in a good
       
   148   general-purpose allocator for malloc-intensive programs.
       
   149 
       
   150   In most ways, this malloc is a best-fit allocator. Generally, it
       
   151   chooses the best-fitting existing chunk for a request, with ties
       
   152   broken in approximately least-recently-used order. (This strategy
       
   153   normally maintains low fragmentation.) However, for requests less
       
   154   than 256bytes, it deviates from best-fit when there is not an
       
   155   exactly fitting available chunk by preferring to use space adjacent
       
   156   to that used for the previous small request, as well as by breaking
       
   157   ties in approximately most-recently-used order. (These enhance
       
   158   locality of series of small allocations.)  And for very large requests
       
   159   (>= 256Kb by default), it relies on system memory mapping
       
   160   facilities, if supported.  (This helps avoid carrying around and
       
   161   possibly fragmenting memory used only for large chunks.)
       
   162 
       
   163   All operations (except malloc_stats and mallinfo) have execution
       
   164   times that are bounded by a constant factor of the number of bits in
       
   165   a size_t, not counting any clearing in calloc or copying in realloc,
       
   166   or actions surrounding MORECORE and MMAP that have times
       
   167   proportional to the number of non-contiguous regions returned by
       
   168   system allocation routines, which is often just 1. In real-time
       
   169   applications, you can optionally suppress segment traversals using
       
   170   NO_SEGMENT_TRAVERSAL, which assures bounded execution even when
       
   171   system allocators return non-contiguous spaces, at the typical
       
   172   expense of carrying around more memory and increased fragmentation.
       
   173 
       
   174   The implementation is not very modular and seriously overuses
       
   175   macros. Perhaps someday all C compilers will do as good a job
       
   176   inlining modular code as can now be done by brute-force expansion,
       
   177   but now, enough of them seem not to.
       
   178 
       
   179   Some compilers issue a lot of warnings about code that is
       
   180   dead/unreachable only on some platforms, and also about intentional
       
   181   uses of negation on unsigned types. All known cases of each can be
       
   182   ignored.
       
   183 
       
   184   For a longer but out of date high-level description, see
       
   185      http://gee.cs.oswego.edu/dl/html/malloc.html
       
   186 
       
   187 * MSPACES
       
   188   If MSPACES is defined, then in addition to malloc, free, etc.,
       
   189   this file also defines mspace_malloc, mspace_free, etc. These
       
   190   are versions of malloc routines that take an "mspace" argument
       
   191   obtained using create_mspace, to control all internal bookkeeping.
       
   192   If ONLY_MSPACES is defined, only these versions are compiled.
       
   193   So if you would like to use this allocator for only some allocations,
       
   194   and your system malloc for others, you can compile with
       
   195   ONLY_MSPACES and then do something like...
       
   196     static mspace mymspace = create_mspace(0,0); // for example
       
   197     #define mymalloc(bytes)  mspace_malloc(mymspace, bytes)
       
   198 
       
   199   (Note: If you only need one instance of an mspace, you can instead
       
   200   use "USE_DL_PREFIX" to relabel the global malloc.)
       
   201 
       
   202   You can similarly create thread-local allocators by storing
       
   203   mspaces as thread-locals. For example:
       
   204     static __thread mspace tlms = 0;
       
   205     void*  tlmalloc(size_t bytes) {
       
   206       if (tlms == 0) tlms = create_mspace(0, 0);
       
   207       return mspace_malloc(tlms, bytes);
       
   208     }
       
   209     void  tlfree(void* mem) { mspace_free(tlms, mem); }
       
   210 
       
   211   Unless FOOTERS is defined, each mspace is completely independent.
       
   212   You cannot allocate from one and free to another (although
       
   213   conformance is only weakly checked, so usage errors are not always
       
   214   caught). If FOOTERS is defined, then each chunk carries around a tag
       
   215   indicating its originating mspace, and frees are directed to their
       
   216   originating spaces.
       
   217 
       
   218  -------------------------  Compile-time options ---------------------------
       
   219 
       
   220 Be careful in setting #define values for numerical constants of type
       
   221 size_t. On some systems, literal values are not automatically extended
       
   222 to size_t precision unless they are explicitly casted. You can also
       
   223 use the symbolic values MAX_SIZE_T, SIZE_T_ONE, etc below.
       
   224 
       
   225 WIN32                    default: defined if _WIN32 defined
       
   226   Defining WIN32 sets up defaults for MS environment and compilers.
       
   227   Otherwise defaults are for unix.
       
   228 
       
   229 MALLOC_ALIGNMENT         default: (size_t)8
       
   230   Controls the minimum alignment for malloc'ed chunks.  It must be a
       
   231   power of two and at least 8, even on machines for which smaller
       
   232   alignments would suffice. It may be defined as larger than this
       
   233   though. Note however that code and data structures are optimized for
       
   234   the case of 8-byte alignment.
       
   235 
       
   236 MSPACES                  default: 0 (false)
       
   237   If true, compile in support for independent allocation spaces.
       
   238   This is only supported if HAVE_MMAP is true.
       
   239 
       
   240 ONLY_MSPACES             default: 0 (false)
       
   241   If true, only compile in mspace versions, not regular versions.
       
   242 
       
   243 USE_LOCKS                default: 0 (false)
       
   244   Causes each call to each public routine to be surrounded with
       
   245   pthread or WIN32 mutex lock/unlock. (If set true, this can be
       
   246   overridden on a per-mspace basis for mspace versions.) If set to a
       
   247   non-zero value other than 1, locks are used, but their
       
   248   implementation is left out, so lock functions must be supplied manually.
       
   249 
       
   250 USE_SPIN_LOCKS           default: 1 iff USE_LOCKS and on x86 using gcc or MSC
       
   251   If true, uses custom spin locks for locking. This is currently
       
   252   supported only for x86 platforms using gcc or recent MS compilers.
       
   253   Otherwise, posix locks or win32 critical sections are used.
       
   254 
       
   255 FOOTERS                  default: 0
       
   256   If true, provide extra checking and dispatching by placing
       
   257   information in the footers of allocated chunks. This adds
       
   258   space and time overhead.
       
   259 
       
   260 INSECURE                 default: 0
       
   261   If true, omit checks for usage errors and heap space overwrites.
       
   262 
       
   263 USE_DL_PREFIX            default: NOT defined
       
   264   Causes compiler to prefix all public routines with the string 'dl'.
       
   265   This can be useful when you only want to use this malloc in one part
       
   266   of a program, using your regular system malloc elsewhere.
       
   267 
       
   268 ABORT                    default: defined as abort()
       
   269   Defines how to abort on failed checks.  On most systems, a failed
       
   270   check cannot die with an "assert" or even print an informative
       
   271   message, because the underlying print routines in turn call malloc,
       
   272   which will fail again.  Generally, the best policy is to simply call
       
   273   abort(). It's not very useful to do more than this because many
       
   274   errors due to overwriting will show up as address faults (null, odd
       
   275   addresses etc) rather than malloc-triggered checks, so will also
       
   276   abort.  Also, most compilers know that abort() does not return, so
       
   277   can better optimize code conditionally calling it.
       
   278 
       
   279 PROCEED_ON_ERROR           default: defined as 0 (false)
       
   280   Controls whether detected bad addresses cause them to bypassed
       
   281   rather than aborting. If set, detected bad arguments to free and
       
   282   realloc are ignored. And all bookkeeping information is zeroed out
       
   283   upon a detected overwrite of freed heap space, thus losing the
       
   284   ability to ever return it from malloc again, but enabling the
       
   285   application to proceed. If PROCEED_ON_ERROR is defined, the
       
   286   static variable malloc_corruption_error_count is compiled in
       
   287   and can be examined to see if errors have occurred. This option
       
   288   generates slower code than the default abort policy.
       
   289 
       
   290 DEBUG                    default: NOT defined
       
   291   The DEBUG setting is mainly intended for people trying to modify
       
   292   this code or diagnose problems when porting to new platforms.
       
   293   However, it may also be able to better isolate user errors than just
       
   294   using runtime checks.  The assertions in the check routines spell
       
   295   out in more detail the assumptions and invariants underlying the
       
   296   algorithms.  The checking is fairly extensive, and will slow down
       
   297   execution noticeably. Calling malloc_stats or mallinfo with DEBUG
       
   298   set will attempt to check every non-mmapped allocated and free chunk
       
   299   in the course of computing the summaries.
       
   300 
       
   301 ABORT_ON_ASSERT_FAILURE   default: defined as 1 (true)
       
   302   Debugging assertion failures can be nearly impossible if your
       
   303   version of the assert macro causes malloc to be called, which will
       
   304   lead to a cascade of further failures, blowing the runtime stack.
       
   305   ABORT_ON_ASSERT_FAILURE cause assertions failures to call abort(),
       
   306   which will usually make debugging easier.
       
   307 
       
   308 MALLOC_FAILURE_ACTION     default: sets errno to ENOMEM, or no-op on win32
       
   309   The action to take before "return 0" when malloc fails to be able to
       
   310   return memory because there is none available.
       
   311 
       
   312 HAVE_MORECORE             default: 1 (true) unless win32 or ONLY_MSPACES
       
   313   True if this system supports sbrk or an emulation of it.
       
   314 
       
   315 MORECORE                  default: sbrk
       
   316   The name of the sbrk-style system routine to call to obtain more
       
   317   memory.  See below for guidance on writing custom MORECORE
       
   318   functions. The type of the argument to sbrk/MORECORE varies across
       
   319   systems.  It cannot be size_t, because it supports negative
       
   320   arguments, so it is normally the signed type of the same width as
       
   321   size_t (sometimes declared as "intptr_t").  It doesn't much matter
       
   322   though. Internally, we only call it with arguments less than half
       
   323   the max value of a size_t, which should work across all reasonable
       
   324   possibilities, although sometimes generating compiler warnings.  See
       
   325   near the end of this file for guidelines for creating a custom
       
   326   version of MORECORE.
       
   327 
       
   328 MORECORE_CONTIGUOUS       default: 1 (true) if HAVE_MORECORE
       
   329   If true, take advantage of fact that consecutive calls to MORECORE
       
   330   with positive arguments always return contiguous increasing
       
   331   addresses.  This is true of unix sbrk. It does not hurt too much to
       
   332   set it true anyway, since malloc copes with non-contiguities.
       
   333   Setting it false when definitely non-contiguous saves time
       
   334   and possibly wasted space it would take to discover this though.
       
   335 
       
   336 MORECORE_CANNOT_TRIM      default: NOT defined
       
   337   True if MORECORE cannot release space back to the system when given
       
   338   negative arguments. This is generally necessary only if you are
       
   339   using a hand-crafted MORECORE function that cannot handle negative
       
   340   arguments.
       
   341 
       
   342 NO_SEGMENT_TRAVERSAL       default: 0
       
   343   If non-zero, suppresses traversals of memory segments
       
   344   returned by either MORECORE or CALL_MMAP. This disables
       
   345   merging of segments that are contiguous, and selectively
       
   346   releasing them to the OS if unused, but bounds execution times.
       
   347 
       
   348 HAVE_MMAP                 default: 1 (true)
       
   349   True if this system supports mmap or an emulation of it.  If so, and
       
   350   HAVE_MORECORE is not true, MMAP is used for all system
       
   351   allocation. If set and HAVE_MORECORE is true as well, MMAP is
       
   352   primarily used to directly allocate very large blocks. It is also
       
   353   used as a backup strategy in cases where MORECORE fails to provide
       
   354   space from system. Note: A single call to MUNMAP is assumed to be
       
   355   able to unmap memory that may have be allocated using multiple calls
       
   356   to MMAP, so long as they are adjacent.
       
   357 
       
   358 HAVE_MREMAP               default: 1 on linux, else 0
       
   359   If true realloc() uses mremap() to re-allocate large blocks and
       
   360   extend or shrink allocation spaces.
       
   361 
       
   362 MMAP_CLEARS               default: 1 except on WINCE.
       
   363   True if mmap clears memory so calloc doesn't need to. This is true
       
   364   for standard unix mmap using /dev/zero and on WIN32 except for WINCE.
       
   365 
       
   366 USE_BUILTIN_FFS            default: 0 (i.e., not used)
       
   367   Causes malloc to use the builtin ffs() function to compute indices.
       
   368   Some compilers may recognize and intrinsify ffs to be faster than the
       
   369   supplied C version. Also, the case of x86 using gcc is special-cased
       
   370   to an asm instruction, so is already as fast as it can be, and so
       
   371   this setting has no effect. Similarly for Win32 under recent MS compilers.
       
   372   (On most x86s, the asm version is only slightly faster than the C version.)
       
   373 
       
   374 malloc_getpagesize         default: derive from system includes, or 4096.
       
   375   The system page size. To the extent possible, this malloc manages
       
   376   memory from the system in page-size units.  This may be (and
       
   377   usually is) a function rather than a constant. This is ignored
       
   378   if WIN32, where page size is determined using getSystemInfo during
       
   379   initialization.
       
   380 
       
   381 USE_DEV_RANDOM             default: 0 (i.e., not used)
       
   382   Causes malloc to use /dev/random to initialize secure magic seed for
       
   383   stamping footers. Otherwise, the current time is used.
       
   384 
       
   385 NO_MALLINFO                default: 0
       
   386   If defined, don't compile "mallinfo". This can be a simple way
       
   387   of dealing with mismatches between system declarations and
       
   388   those in this file.
       
   389 
       
   390 MALLINFO_FIELD_TYPE        default: size_t
       
   391   The type of the fields in the mallinfo struct. This was originally
       
   392   defined as "int" in SVID etc, but is more usefully defined as
       
   393   size_t. The value is used only if  HAVE_USR_INCLUDE_MALLOC_H is not set
       
   394 
       
   395 REALLOC_ZERO_BYTES_FREES    default: not defined
       
   396   This should be set if a call to realloc with zero bytes should
       
   397   be the same as a call to free. Some people think it should. Otherwise,
       
   398   since this malloc returns a unique pointer for malloc(0), so does
       
   399   realloc(p, 0).
       
   400 
       
   401 LACKS_UNISTD_H, LACKS_FCNTL_H, LACKS_SYS_PARAM_H, LACKS_SYS_MMAN_H
       
   402 LACKS_STRINGS_H, LACKS_STRING_H, LACKS_SYS_TYPES_H,  LACKS_ERRNO_H
       
   403 LACKS_STDLIB_H                default: NOT defined unless on WIN32
       
   404   Define these if your system does not have these header files.
       
   405   You might need to manually insert some of the declarations they provide.
       
   406 
       
   407 DEFAULT_GRANULARITY        default: page size if MORECORE_CONTIGUOUS,
       
   408                                 system_info.dwAllocationGranularity in WIN32,
       
   409                                 otherwise 64K.
       
   410       Also settable using mallopt(M_GRANULARITY, x)
       
   411   The unit for allocating and deallocating memory from the system.  On
       
   412   most systems with contiguous MORECORE, there is no reason to
       
   413   make this more than a page. However, systems with MMAP tend to
       
   414   either require or encourage larger granularities.  You can increase
       
   415   this value to prevent system allocation functions to be called so
       
   416   often, especially if they are slow.  The value must be at least one
       
   417   page and must be a power of two.  Setting to 0 causes initialization
       
   418   to either page size or win32 region size.  (Note: In previous
       
   419   versions of malloc, the equivalent of this option was called
       
   420   "TOP_PAD")
       
   421 
       
   422 DEFAULT_TRIM_THRESHOLD    default: 2MB
       
   423       Also settable using mallopt(M_TRIM_THRESHOLD, x)
       
   424   The maximum amount of unused top-most memory to keep before
       
   425   releasing via malloc_trim in free().  Automatic trimming is mainly
       
   426   useful in long-lived programs using contiguous MORECORE.  Because
       
   427   trimming via sbrk can be slow on some systems, and can sometimes be
       
   428   wasteful (in cases where programs immediately afterward allocate
       
   429   more large chunks) the value should be high enough so that your
       
   430   overall system performance would improve by releasing this much
       
   431   memory.  As a rough guide, you might set to a value close to the
       
   432   average size of a process (program) running on your system.
       
   433   Releasing this much memory would allow such a process to run in
       
   434   memory.  Generally, it is worth tuning trim thresholds when a
       
   435   program undergoes phases where several large chunks are allocated
       
   436   and released in ways that can reuse each other's storage, perhaps
       
   437   mixed with phases where there are no such chunks at all. The trim
       
   438   value must be greater than page size to have any useful effect.  To
       
   439   disable trimming completely, you can set to MAX_SIZE_T. Note that the trick
       
   440   some people use of mallocing a huge space and then freeing it at
       
   441   program startup, in an attempt to reserve system memory, doesn't
       
   442   have the intended effect under automatic trimming, since that memory
       
   443   will immediately be returned to the system.
       
   444 
       
   445 DEFAULT_MMAP_THRESHOLD       default: 256K
       
   446       Also settable using mallopt(M_MMAP_THRESHOLD, x)
       
   447   The request size threshold for using MMAP to directly service a
       
   448   request. Requests of at least this size that cannot be allocated
       
   449   using already-existing space will be serviced via mmap.  (If enough
       
   450   normal freed space already exists it is used instead.)  Using mmap
       
   451   segregates relatively large chunks of memory so that they can be
       
   452   individually obtained and released from the host system. A request
       
   453   serviced through mmap is never reused by any other request (at least
       
   454   not directly; the system may just so happen to remap successive
       
   455   requests to the same locations).  Segregating space in this way has
       
   456   the benefits that: Mmapped space can always be individually released
       
   457   back to the system, which helps keep the system level memory demands
       
   458   of a long-lived program low.  Also, mapped memory doesn't become
       
   459   `locked' between other chunks, as can happen with normally allocated
       
   460   chunks, which means that even trimming via malloc_trim would not
       
   461   release them.  However, it has the disadvantage that the space
       
   462   cannot be reclaimed, consolidated, and then used to service later
       
   463   requests, as happens with normal chunks.  The advantages of mmap
       
   464   nearly always outweigh disadvantages for "large" chunks, but the
       
   465   value of "large" may vary across systems.  The default is an
       
   466   empirically derived value that works well in most systems. You can
       
   467   disable mmap by setting to MAX_SIZE_T.
       
   468 
       
   469 MAX_RELEASE_CHECK_RATE   default: 255 unless not HAVE_MMAP
       
   470   The number of consolidated frees between checks to release
       
   471   unused segments when freeing. When using non-contiguous segments,
       
   472   especially with multiple mspaces, checking only for topmost space
       
   473   doesn't always suffice to trigger trimming. To compensate for this,
       
   474   free() will, with a period of MAX_RELEASE_CHECK_RATE (or the
       
   475   current number of segments, if greater) try to release unused
       
   476   segments to the OS when freeing chunks that result in
       
   477   consolidation. The best value for this parameter is a compromise
       
   478   between slowing down frees with relatively costly checks that
       
   479   rarely trigger versus holding on to unused memory. To effectively
       
   480   disable, set to MAX_SIZE_T. This may lead to a very slight speed
       
   481   improvement at the expense of carrying around more memory.
       
   482 */
       
   483 
       
   484 #ifndef WIN32
       
   485 #ifdef _WIN32
       
   486 #define WIN32 1
       
   487 #endif  /* _WIN32 */
       
   488 #endif  /* WIN32 */
       
   489 #ifdef WIN32
       
   490 #define WIN32_LEAN_AND_MEAN
       
   491 #include <windows.h>
       
   492 #define HAVE_MMAP 1
       
   493 #define HAVE_MORECORE 0
       
   494 #define LACKS_UNISTD_H
       
   495 #define LACKS_SYS_PARAM_H
       
   496 #define LACKS_SYS_MMAN_H
       
   497 #define LACKS_STRING_H
       
   498 #define LACKS_STRINGS_H
       
   499 #define LACKS_SYS_TYPES_H
       
   500 #define LACKS_ERRNO_H
       
   501 #define MALLOC_FAILURE_ACTION
       
   502 #ifdef _WIN32_WCE /* WINCE reportedly does not clear */
       
   503 #define MMAP_CLEARS 0
       
   504 #else
       
   505 #define MMAP_CLEARS 1
       
   506 #endif /* _WIN32_WCE */
       
   507 #endif  /* WIN32 */
       
   508 
       
   509 #if defined(DARWIN) || defined(_DARWIN)
       
   510 /* Mac OSX docs advise not to use sbrk; it seems better to use mmap */
       
   511 #ifndef HAVE_MORECORE
       
   512 #define HAVE_MORECORE 0
       
   513 #define HAVE_MMAP 1
       
   514 #endif  /* HAVE_MORECORE */
       
   515 #endif  /* DARWIN */
       
   516 
       
   517 #ifndef LACKS_SYS_TYPES_H
       
   518 #include <sys/types.h>  /* For size_t */
       
   519 #endif  /* LACKS_SYS_TYPES_H */
       
   520 
       
   521 /* The maximum possible size_t value has all bits set */
       
   522 #define MAX_SIZE_T           (~(size_t)0)
       
   523 
       
   524 #ifndef ONLY_MSPACES
       
   525 #define ONLY_MSPACES 0
       
   526 #endif  /* ONLY_MSPACES */
       
   527 #ifndef MSPACES
       
   528 #if ONLY_MSPACES
       
   529 #define MSPACES 1
       
   530 #else   /* ONLY_MSPACES */
       
   531 #define MSPACES 0
       
   532 #endif  /* ONLY_MSPACES */
       
   533 #endif  /* MSPACES */
       
   534 #ifndef MALLOC_ALIGNMENT
       
   535 #define MALLOC_ALIGNMENT ((size_t)8U)
       
   536 #endif  /* MALLOC_ALIGNMENT */
       
   537 #ifndef FOOTERS
       
   538 #define FOOTERS 0
       
   539 #endif  /* FOOTERS */
       
   540 #ifndef ABORT
       
   541 #define ABORT  abort()
       
   542 #endif  /* ABORT */
       
   543 #ifndef ABORT_ON_ASSERT_FAILURE
       
   544 #define ABORT_ON_ASSERT_FAILURE 1
       
   545 #endif  /* ABORT_ON_ASSERT_FAILURE */
       
   546 #ifndef PROCEED_ON_ERROR
       
   547 #define PROCEED_ON_ERROR 0
       
   548 #endif  /* PROCEED_ON_ERROR */
       
   549 #ifndef USE_LOCKS
       
   550 #define USE_LOCKS 0
       
   551 #endif  /* USE_LOCKS */
       
   552 #ifndef USE_SPIN_LOCKS
       
   553 #if USE_LOCKS && (defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))) || (defined(_MSC_VER) && _MSC_VER>=1310)
       
   554 #define USE_SPIN_LOCKS 1
       
   555 #else
       
   556 #define USE_SPIN_LOCKS 0
       
   557 #endif /* USE_LOCKS && ... */
       
   558 #endif /* USE_SPIN_LOCKS */
       
   559 #ifndef INSECURE
       
   560 #define INSECURE 0
       
   561 #endif  /* INSECURE */
       
   562 #ifndef HAVE_MMAP
       
   563 #define HAVE_MMAP 1
       
   564 #endif  /* HAVE_MMAP */
       
   565 #ifndef MMAP_CLEARS
       
   566 #define MMAP_CLEARS 1
       
   567 #endif  /* MMAP_CLEARS */
       
   568 #ifndef HAVE_MREMAP
       
   569 #ifdef linux
       
   570 #define HAVE_MREMAP 1
       
   571 #else   /* linux */
       
   572 #define HAVE_MREMAP 0
       
   573 #endif  /* linux */
       
   574 #endif  /* HAVE_MREMAP */
       
   575 #ifndef MALLOC_FAILURE_ACTION
       
   576 #define MALLOC_FAILURE_ACTION  errno = ENOMEM;
       
   577 #endif  /* MALLOC_FAILURE_ACTION */
       
   578 #ifndef HAVE_MORECORE
       
   579 #if ONLY_MSPACES
       
   580 #define HAVE_MORECORE 0
       
   581 #else   /* ONLY_MSPACES */
       
   582 #define HAVE_MORECORE 1
       
   583 #endif  /* ONLY_MSPACES */
       
   584 #endif  /* HAVE_MORECORE */
       
   585 #if !HAVE_MORECORE
       
   586 #define MORECORE_CONTIGUOUS 0
       
   587 #else   /* !HAVE_MORECORE */
       
   588 #ifndef MORECORE
       
   589 #define MORECORE sbrk
       
   590 #endif  /* MORECORE */
       
   591 #ifndef MORECORE_CONTIGUOUS
       
   592 #define MORECORE_CONTIGUOUS 1
       
   593 #endif  /* MORECORE_CONTIGUOUS */
       
   594 #endif  /* HAVE_MORECORE */
       
   595 #ifndef DEFAULT_GRANULARITY
       
   596 #if MORECORE_CONTIGUOUS
       
   597 #define DEFAULT_GRANULARITY (0)  /* 0 means to compute in init_mparams */
       
   598 #else   /* MORECORE_CONTIGUOUS */
       
   599 #define DEFAULT_GRANULARITY ((size_t)64U * (size_t)1024U)
       
   600 #endif  /* MORECORE_CONTIGUOUS */
       
   601 #endif  /* DEFAULT_GRANULARITY */
       
   602 #ifndef DEFAULT_TRIM_THRESHOLD
       
   603 #ifndef MORECORE_CANNOT_TRIM
       
   604 #define DEFAULT_TRIM_THRESHOLD ((size_t)2U * (size_t)1024U * (size_t)1024U)
       
   605 #else   /* MORECORE_CANNOT_TRIM */
       
   606 #define DEFAULT_TRIM_THRESHOLD MAX_SIZE_T
       
   607 #endif  /* MORECORE_CANNOT_TRIM */
       
   608 #endif  /* DEFAULT_TRIM_THRESHOLD */
       
   609 #ifndef DEFAULT_MMAP_THRESHOLD
       
   610 #if HAVE_MMAP
       
   611 #define DEFAULT_MMAP_THRESHOLD ((size_t)256U * (size_t)1024U)
       
   612 #else   /* HAVE_MMAP */
       
   613 #define DEFAULT_MMAP_THRESHOLD MAX_SIZE_T
       
   614 #endif  /* HAVE_MMAP */
       
   615 #endif  /* DEFAULT_MMAP_THRESHOLD */
       
   616 #ifndef MAX_RELEASE_CHECK_RATE
       
   617 #if HAVE_MMAP
       
   618 #define MAX_RELEASE_CHECK_RATE 255
       
   619 #else
       
   620 #define MAX_RELEASE_CHECK_RATE MAX_SIZE_T
       
   621 #endif /* HAVE_MMAP */
       
   622 #endif /* MAX_RELEASE_CHECK_RATE */
       
   623 #ifndef USE_BUILTIN_FFS
       
   624 #define USE_BUILTIN_FFS 0
       
   625 #endif  /* USE_BUILTIN_FFS */
       
   626 #ifndef USE_DEV_RANDOM
       
   627 #define USE_DEV_RANDOM 0
       
   628 #endif  /* USE_DEV_RANDOM */
       
   629 #ifndef NO_MALLINFO
       
   630 #define NO_MALLINFO 0
       
   631 #endif  /* NO_MALLINFO */
       
   632 #ifndef MALLINFO_FIELD_TYPE
       
   633 #define MALLINFO_FIELD_TYPE size_t
       
   634 #endif  /* MALLINFO_FIELD_TYPE */
       
   635 #ifndef NO_SEGMENT_TRAVERSAL
       
   636 #define NO_SEGMENT_TRAVERSAL 0
       
   637 #endif /* NO_SEGMENT_TRAVERSAL */
       
   638 
       
   639 /*
       
   640   mallopt tuning options.  SVID/XPG defines four standard parameter
       
   641   numbers for mallopt, normally defined in malloc.h.  None of these
       
   642   are used in this malloc, so setting them has no effect. But this
       
   643   malloc does support the following options.
       
   644 */
       
   645 
       
   646 #define M_TRIM_THRESHOLD     (-1)
       
   647 #define M_GRANULARITY        (-2)
       
   648 #define M_MMAP_THRESHOLD     (-3)
       
   649 
       
   650 /* ------------------------ Mallinfo declarations ------------------------ */
       
   651 
       
   652 #if !NO_MALLINFO
       
   653 /*
       
   654   This version of malloc supports the standard SVID/XPG mallinfo
       
   655   routine that returns a struct containing usage properties and
       
   656   statistics. It should work on any system that has a
       
   657   /usr/include/malloc.h defining struct mallinfo.  The main
       
   658   declaration needed is the mallinfo struct that is returned (by-copy)
       
   659   by mallinfo().  The malloinfo struct contains a bunch of fields that
       
   660   are not even meaningful in this version of malloc.  These fields are
       
   661   are instead filled by mallinfo() with other numbers that might be of
       
   662   interest.
       
   663 
       
   664   HAVE_USR_INCLUDE_MALLOC_H should be set if you have a
       
   665   /usr/include/malloc.h file that includes a declaration of struct
       
   666   mallinfo.  If so, it is included; else a compliant version is
       
   667   declared below.  These must be precisely the same for mallinfo() to
       
   668   work.  The original SVID version of this struct, defined on most
       
   669   systems with mallinfo, declares all fields as ints. But some others
       
   670   define as unsigned long. If your system defines the fields using a
       
   671   type of different width than listed here, you MUST #include your
       
   672   system version and #define HAVE_USR_INCLUDE_MALLOC_H.
       
   673 */
       
   674 
       
   675 /* #define HAVE_USR_INCLUDE_MALLOC_H */
       
   676 
       
   677 #ifdef HAVE_USR_INCLUDE_MALLOC_H
       
   678 #include "/usr/include/malloc.h"
       
   679 #else /* HAVE_USR_INCLUDE_MALLOC_H */
       
   680 
       
   681 struct mallinfo {
       
   682   MALLINFO_FIELD_TYPE arena;    /* non-mmapped space allocated from system */
       
   683   MALLINFO_FIELD_TYPE ordblks;  /* number of free chunks */
       
   684   MALLINFO_FIELD_TYPE smblks;   /* always 0 */
       
   685   MALLINFO_FIELD_TYPE hblks;    /* always 0 */
       
   686   MALLINFO_FIELD_TYPE hblkhd;   /* space in mmapped regions */
       
   687   MALLINFO_FIELD_TYPE usmblks;  /* maximum total allocated space */
       
   688   MALLINFO_FIELD_TYPE fsmblks;  /* always 0 */
       
   689   MALLINFO_FIELD_TYPE uordblks; /* total allocated space */
       
   690   MALLINFO_FIELD_TYPE fordblks; /* total free space */
       
   691   MALLINFO_FIELD_TYPE keepcost; /* releasable (via malloc_trim) space */
       
   692 };
       
   693 
       
   694 #endif /* HAVE_USR_INCLUDE_MALLOC_H */
       
   695 #endif /* NO_MALLINFO */
       
   696 
       
   697 /*
       
   698   Try to persuade compilers to inline. The most critical functions for
       
   699   inlining are defined as macros, so these aren't used for them.
       
   700 */
       
   701 
       
   702 #ifndef FORCEINLINE
       
   703   #if defined(__GNUC__)
       
   704 #define FORCEINLINE __inline __attribute__ ((always_inline))
       
   705   #elif defined(_MSC_VER)
       
   706     #define FORCEINLINE __forceinline
       
   707   #endif
       
   708 #endif
       
   709 #ifndef NOINLINE
       
   710   #if defined(__GNUC__)
       
   711     #define NOINLINE __attribute__ ((noinline))
       
   712   #elif defined(_MSC_VER)
       
   713     #define NOINLINE __declspec(noinline)
       
   714   #else
       
   715     #define NOINLINE
       
   716   #endif
       
   717 #endif
       
   718 
       
   719 #ifdef __cplusplus
       
   720 extern "C" {
       
   721 #ifndef FORCEINLINE
       
   722  #define FORCEINLINE inline
       
   723 #endif
       
   724 #endif /* __cplusplus */
       
   725 #ifndef FORCEINLINE
       
   726  #define FORCEINLINE
       
   727 #endif
       
   728 
       
   729 #if !ONLY_MSPACES
       
   730 
       
   731 /* ------------------- Declarations of public routines ------------------- */
       
   732 
       
   733 #ifndef USE_DL_PREFIX
       
   734 #define dlcalloc               calloc
       
   735 #define dlfree                 free
       
   736 #define dlmalloc               malloc
       
   737 #define dlmemalign             memalign
       
   738 #define dlrealloc              realloc
       
   739 #define dlvalloc               valloc
       
   740 #define dlpvalloc              pvalloc
       
   741 #define dlmallinfo             mallinfo
       
   742 #define dlmallopt              mallopt
       
   743 #define dlmalloc_trim          malloc_trim
       
   744 #define dlmalloc_stats         malloc_stats
       
   745 #define dlmalloc_usable_size   malloc_usable_size
       
   746 #define dlmalloc_footprint     malloc_footprint
       
   747 #define dlmalloc_max_footprint malloc_max_footprint
       
   748 #define dlindependent_calloc   independent_calloc
       
   749 #define dlindependent_comalloc independent_comalloc
       
   750 #endif /* USE_DL_PREFIX */
       
   751 
       
   752 
       
   753 /*
       
   754   malloc(size_t n)
       
   755   Returns a pointer to a newly allocated chunk of at least n bytes, or
       
   756   null if no space is available, in which case errno is set to ENOMEM
       
   757   on ANSI C systems.
       
   758 
       
   759   If n is zero, malloc returns a minimum-sized chunk. (The minimum
       
   760   size is 16 bytes on most 32bit systems, and 32 bytes on 64bit
       
   761   systems.)  Note that size_t is an unsigned type, so calls with
       
   762   arguments that would be negative if signed are interpreted as
       
   763   requests for huge amounts of space, which will often fail. The
       
   764   maximum supported value of n differs across systems, but is in all
       
   765   cases less than the maximum representable value of a size_t.
       
   766 */
       
   767 void* dlmalloc(size_t);
       
   768 
       
   769 /*
       
   770   free(void* p)
       
   771   Releases the chunk of memory pointed to by p, that had been previously
       
   772   allocated using malloc or a related routine such as realloc.
       
   773   It has no effect if p is null. If p was not malloced or already
       
   774   freed, free(p) will by default cause the current program to abort.
       
   775 */
       
   776 void  dlfree(void*);
       
   777 
       
   778 /*
       
   779   calloc(size_t n_elements, size_t element_size);
       
   780   Returns a pointer to n_elements * element_size bytes, with all locations
       
   781   set to zero.
       
   782 */
       
   783 void* dlcalloc(size_t, size_t);
       
   784 
       
   785 /*
       
   786   realloc(void* p, size_t n)
       
   787   Returns a pointer to a chunk of size n that contains the same data
       
   788   as does chunk p up to the minimum of (n, p's size) bytes, or null
       
   789   if no space is available.
       
   790 
       
   791   The returned pointer may or may not be the same as p. The algorithm
       
   792   prefers extending p in most cases when possible, otherwise it
       
   793   employs the equivalent of a malloc-copy-free sequence.
       
   794 
       
   795   If p is null, realloc is equivalent to malloc.
       
   796 
       
   797   If space is not available, realloc returns null, errno is set (if on
       
   798   ANSI) and p is NOT freed.
       
   799 
       
   800   if n is for fewer bytes than already held by p, the newly unused
       
   801   space is lopped off and freed if possible.  realloc with a size
       
   802   argument of zero (re)allocates a minimum-sized chunk.
       
   803 
       
   804   The old unix realloc convention of allowing the last-free'd chunk
       
   805   to be used as an argument to realloc is not supported.
       
   806 */
       
   807 
       
   808 void* dlrealloc(void*, size_t);
       
   809 
       
   810 /*
       
   811   memalign(size_t alignment, size_t n);
       
   812   Returns a pointer to a newly allocated chunk of n bytes, aligned
       
   813   in accord with the alignment argument.
       
   814 
       
   815   The alignment argument should be a power of two. If the argument is
       
   816   not a power of two, the nearest greater power is used.
       
   817   8-byte alignment is guaranteed by normal malloc calls, so don't
       
   818   bother calling memalign with an argument of 8 or less.
       
   819 
       
   820   Overreliance on memalign is a sure way to fragment space.
       
   821 */
       
   822 void* dlmemalign(size_t, size_t);
       
   823 
       
   824 /*
       
   825   valloc(size_t n);
       
   826   Equivalent to memalign(pagesize, n), where pagesize is the page
       
   827   size of the system. If the pagesize is unknown, 4096 is used.
       
   828 */
       
   829 void* dlvalloc(size_t);
       
   830 
       
   831 /*
       
   832   mallopt(int parameter_number, int parameter_value)
       
   833   Sets tunable parameters The format is to provide a
       
   834   (parameter-number, parameter-value) pair.  mallopt then sets the
       
   835   corresponding parameter to the argument value if it can (i.e., so
       
   836   long as the value is meaningful), and returns 1 if successful else
       
   837   0.  SVID/XPG/ANSI defines four standard param numbers for mallopt,
       
   838   normally defined in malloc.h.  None of these are use in this malloc,
       
   839   so setting them has no effect. But this malloc also supports other
       
   840   options in mallopt. See below for details.  Briefly, supported
       
   841   parameters are as follows (listed defaults are for "typical"
       
   842   configurations).
       
   843 
       
   844   Symbol            param #  default    allowed param values
       
   845   M_TRIM_THRESHOLD     -1   2*1024*1024   any   (MAX_SIZE_T disables)
       
   846   M_GRANULARITY        -2     page size   any power of 2 >= page size
       
   847   M_MMAP_THRESHOLD     -3      256*1024   any   (or 0 if no MMAP support)
       
   848 */
       
   849 int dlmallopt(int, int);
       
   850 
       
   851 /*
       
   852   malloc_footprint();
       
   853   Returns the number of bytes obtained from the system.  The total
       
   854   number of bytes allocated by malloc, realloc etc., is less than this
       
   855   value. Unlike mallinfo, this function returns only a precomputed
       
   856   result, so can be called frequently to monitor memory consumption.
       
   857   Even if locks are otherwise defined, this function does not use them,
       
   858   so results might not be up to date.
       
   859 */
       
   860 size_t dlmalloc_footprint(void);
       
   861 
       
   862 /*
       
   863   malloc_max_footprint();
       
   864   Returns the maximum number of bytes obtained from the system. This
       
   865   value will be greater than current footprint if deallocated space
       
   866   has been reclaimed by the system. The peak number of bytes allocated
       
   867   by malloc, realloc etc., is less than this value. Unlike mallinfo,
       
   868   this function returns only a precomputed result, so can be called
       
   869   frequently to monitor memory consumption.  Even if locks are
       
   870   otherwise defined, this function does not use them, so results might
       
   871   not be up to date.
       
   872 */
       
   873 size_t dlmalloc_max_footprint(void);
       
   874 
       
   875 #if !NO_MALLINFO
       
   876 /*
       
   877   mallinfo()
       
   878   Returns (by copy) a struct containing various summary statistics:
       
   879 
       
   880   arena:     current total non-mmapped bytes allocated from system
       
   881   ordblks:   the number of free chunks
       
   882   smblks:    always zero.
       
   883   hblks:     current number of mmapped regions
       
   884   hblkhd:    total bytes held in mmapped regions
       
   885   usmblks:   the maximum total allocated space. This will be greater
       
   886                 than current total if trimming has occurred.
       
   887   fsmblks:   always zero
       
   888   uordblks:  current total allocated space (normal or mmapped)
       
   889   fordblks:  total free space
       
   890   keepcost:  the maximum number of bytes that could ideally be released
       
   891                back to system via malloc_trim. ("ideally" means that
       
   892                it ignores page restrictions etc.)
       
   893 
       
   894   Because these fields are ints, but internal bookkeeping may
       
   895   be kept as longs, the reported values may wrap around zero and
       
   896   thus be inaccurate.
       
   897 */
       
   898 struct mallinfo dlmallinfo(void);
       
   899 #endif /* NO_MALLINFO */
       
   900 
       
   901 /*
       
   902   independent_calloc(size_t n_elements, size_t element_size, void* chunks[]);
       
   903 
       
   904   independent_calloc is similar to calloc, but instead of returning a
       
   905   single cleared space, it returns an array of pointers to n_elements
       
   906   independent elements that can hold contents of size elem_size, each
       
   907   of which starts out cleared, and can be independently freed,
       
   908   realloc'ed etc. The elements are guaranteed to be adjacently
       
   909   allocated (this is not guaranteed to occur with multiple callocs or
       
   910   mallocs), which may also improve cache locality in some
       
   911   applications.
       
   912 
       
   913   The "chunks" argument is optional (i.e., may be null, which is
       
   914   probably the most typical usage). If it is null, the returned array
       
   915   is itself dynamically allocated and should also be freed when it is
       
   916   no longer needed. Otherwise, the chunks array must be of at least
       
   917   n_elements in length. It is filled in with the pointers to the
       
   918   chunks.
       
   919 
       
   920   In either case, independent_calloc returns this pointer array, or
       
   921   null if the allocation failed.  If n_elements is zero and "chunks"
       
   922   is null, it returns a chunk representing an array with zero elements
       
   923   (which should be freed if not wanted).
       
   924 
       
   925   Each element must be individually freed when it is no longer
       
   926   needed. If you'd like to instead be able to free all at once, you
       
   927   should instead use regular calloc and assign pointers into this
       
   928   space to represent elements.  (In this case though, you cannot
       
   929   independently free elements.)
       
   930 
       
   931   independent_calloc simplifies and speeds up implementations of many
       
   932   kinds of pools.  It may also be useful when constructing large data
       
   933   structures that initially have a fixed number of fixed-sized nodes,
       
   934   but the number is not known at compile time, and some of the nodes
       
   935   may later need to be freed. For example:
       
   936 
       
   937   struct Node { int item; struct Node* next; };
       
   938 
       
   939   struct Node* build_list() {
       
   940     struct Node** pool;
       
   941     int n = read_number_of_nodes_needed();
       
   942     if (n <= 0) return 0;
       
   943     pool = (struct Node**)(independent_calloc(n, sizeof(struct Node), 0);
       
   944     if (pool == 0) die();
       
   945     // organize into a linked list...
       
   946     struct Node* first = pool[0];
       
   947     for (i = 0; i < n-1; ++i)
       
   948       pool[i]->next = pool[i+1];
       
   949     free(pool);     // Can now free the array (or not, if it is needed later)
       
   950     return first;
       
   951   }
       
   952 */
       
   953 void** dlindependent_calloc(size_t, size_t, void**);
       
   954 
       
   955 /*
       
   956   independent_comalloc(size_t n_elements, size_t sizes[], void* chunks[]);
       
   957 
       
   958   independent_comalloc allocates, all at once, a set of n_elements
       
   959   chunks with sizes indicated in the "sizes" array.    It returns
       
   960   an array of pointers to these elements, each of which can be
       
   961   independently freed, realloc'ed etc. The elements are guaranteed to
       
   962   be adjacently allocated (this is not guaranteed to occur with
       
   963   multiple callocs or mallocs), which may also improve cache locality
       
   964   in some applications.
       
   965 
       
   966   The "chunks" argument is optional (i.e., may be null). If it is null
       
   967   the returned array is itself dynamically allocated and should also
       
   968   be freed when it is no longer needed. Otherwise, the chunks array
       
   969   must be of at least n_elements in length. It is filled in with the
       
   970   pointers to the chunks.
       
   971 
       
   972   In either case, independent_comalloc returns this pointer array, or
       
   973   null if the allocation failed.  If n_elements is zero and chunks is
       
   974   null, it returns a chunk representing an array with zero elements
       
   975   (which should be freed if not wanted).
       
   976 
       
   977   Each element must be individually freed when it is no longer
       
   978   needed. If you'd like to instead be able to free all at once, you
       
   979   should instead use a single regular malloc, and assign pointers at
       
   980   particular offsets in the aggregate space. (In this case though, you
       
   981   cannot independently free elements.)
       
   982 
       
   983   independent_comallac differs from independent_calloc in that each
       
   984   element may have a different size, and also that it does not
       
   985   automatically clear elements.
       
   986 
       
   987   independent_comalloc can be used to speed up allocation in cases
       
   988   where several structs or objects must always be allocated at the
       
   989   same time.  For example:
       
   990 
       
   991   struct Head { ... }
       
   992   struct Foot { ... }
       
   993 
       
   994   void send_message(char* msg) {
       
   995     int msglen = strlen(msg);
       
   996     size_t sizes[3] = { sizeof(struct Head), msglen, sizeof(struct Foot) };
       
   997     void* chunks[3];
       
   998     if (independent_comalloc(3, sizes, chunks) == 0)
       
   999       die();
       
  1000     struct Head* head = (struct Head*)(chunks[0]);
       
  1001     char*        body = (char*)(chunks[1]);
       
  1002     struct Foot* foot = (struct Foot*)(chunks[2]);
       
  1003     // ...
       
  1004   }
       
  1005 
       
  1006   In general though, independent_comalloc is worth using only for
       
  1007   larger values of n_elements. For small values, you probably won't
       
  1008   detect enough difference from series of malloc calls to bother.
       
  1009 
       
  1010   Overuse of independent_comalloc can increase overall memory usage,
       
  1011   since it cannot reuse existing noncontiguous small chunks that
       
  1012   might be available for some of the elements.
       
  1013 */
       
  1014 void** dlindependent_comalloc(size_t, size_t*, void**);
       
  1015 
       
  1016 
       
  1017 /*
       
  1018   pvalloc(size_t n);
       
  1019   Equivalent to valloc(minimum-page-that-holds(n)), that is,
       
  1020   round up n to nearest pagesize.
       
  1021  */
       
  1022 void*  dlpvalloc(size_t);
       
  1023 
       
  1024 /*
       
  1025   malloc_trim(size_t pad);
       
  1026 
       
  1027   If possible, gives memory back to the system (via negative arguments
       
  1028   to sbrk) if there is unused memory at the `high' end of the malloc
       
  1029   pool or in unused MMAP segments. You can call this after freeing
       
  1030   large blocks of memory to potentially reduce the system-level memory
       
  1031   requirements of a program. However, it cannot guarantee to reduce
       
  1032   memory. Under some allocation patterns, some large free blocks of
       
  1033   memory will be locked between two used chunks, so they cannot be
       
  1034   given back to the system.
       
  1035 
       
  1036   The `pad' argument to malloc_trim represents the amount of free
       
  1037   trailing space to leave untrimmed. If this argument is zero, only
       
  1038   the minimum amount of memory to maintain internal data structures
       
  1039   will be left. Non-zero arguments can be supplied to maintain enough
       
  1040   trailing space to service future expected allocations without having
       
  1041   to re-obtain memory from the system.
       
  1042 
       
  1043   Malloc_trim returns 1 if it actually released any memory, else 0.
       
  1044 */
       
  1045 int  dlmalloc_trim(size_t);
       
  1046 
       
  1047 /*
       
  1048   malloc_usable_size(void* p);
       
  1049 
       
  1050   Returns the number of bytes you can actually use in
       
  1051   an allocated chunk, which may be more than you requested (although
       
  1052   often not) due to alignment and minimum size constraints.
       
  1053   You can use this many bytes without worrying about
       
  1054   overwriting other allocated objects. This is not a particularly great
       
  1055   programming practice. malloc_usable_size can be more useful in
       
  1056   debugging and assertions, for example:
       
  1057 
       
  1058   p = malloc(n);
       
  1059   assert(malloc_usable_size(p) >= 256);
       
  1060 */
       
  1061 size_t dlmalloc_usable_size(void*);
       
  1062 
       
  1063 /*
       
  1064   malloc_stats();
       
  1065   Prints on stderr the amount of space obtained from the system (both
       
  1066   via sbrk and mmap), the maximum amount (which may be more than
       
  1067   current if malloc_trim and/or munmap got called), and the current
       
  1068   number of bytes allocated via malloc (or realloc, etc) but not yet
       
  1069   freed. Note that this is the number of bytes allocated, not the
       
  1070   number requested. It will be larger than the number requested
       
  1071   because of alignment and bookkeeping overhead. Because it includes
       
  1072   alignment wastage as being in use, this figure may be greater than
       
  1073   zero even when no user-level chunks are allocated.
       
  1074 
       
  1075   The reported current and maximum system memory can be inaccurate if
       
  1076   a program makes other calls to system memory allocation functions
       
  1077   (normally sbrk) outside of malloc.
       
  1078 
       
  1079   malloc_stats prints only the most commonly interesting statistics.
       
  1080   More information can be obtained by calling mallinfo.
       
  1081 */
       
  1082 void  dlmalloc_stats(void);
       
  1083 
       
  1084 #endif /* ONLY_MSPACES */
       
  1085 
       
  1086 #if MSPACES
       
  1087 
       
  1088 /*
       
  1089   mspace is an opaque type representing an independent
       
  1090   region of space that supports mspace_malloc, etc.
       
  1091 */
       
  1092 typedef void* mspace;
       
  1093 
       
  1094 /*
       
  1095   create_mspace creates and returns a new independent space with the
       
  1096   given initial capacity, or, if 0, the default granularity size.  It
       
  1097   returns null if there is no system memory available to create the
       
  1098   space.  If argument locked is non-zero, the space uses a separate
       
  1099   lock to control access. The capacity of the space will grow
       
  1100   dynamically as needed to service mspace_malloc requests.  You can
       
  1101   control the sizes of incremental increases of this space by
       
  1102   compiling with a different DEFAULT_GRANULARITY or dynamically
       
  1103   setting with mallopt(M_GRANULARITY, value).
       
  1104 */
       
  1105 mspace create_mspace(size_t capacity, int locked);
       
  1106 
       
  1107 /*
       
  1108   destroy_mspace destroys the given space, and attempts to return all
       
  1109   of its memory back to the system, returning the total number of
       
  1110   bytes freed. After destruction, the results of access to all memory
       
  1111   used by the space become undefined.
       
  1112 */
       
  1113 size_t destroy_mspace(mspace msp);
       
  1114 
       
  1115 /*
       
  1116   create_mspace_with_base uses the memory supplied as the initial base
       
  1117   of a new mspace. Part (less than 128*sizeof(size_t) bytes) of this
       
  1118   space is used for bookkeeping, so the capacity must be at least this
       
  1119   large. (Otherwise 0 is returned.) When this initial space is
       
  1120   exhausted, additional memory will be obtained from the system.
       
  1121   Destroying this space will deallocate all additionally allocated
       
  1122   space (if possible) but not the initial base.
       
  1123 */
       
  1124 mspace create_mspace_with_base(void* base, size_t capacity, int locked);
       
  1125 
       
  1126 /*
       
  1127   mspace_malloc behaves as malloc, but operates within
       
  1128   the given space.
       
  1129 */
       
  1130 void* mspace_malloc(mspace msp, size_t bytes);
       
  1131 
       
  1132 /*
       
  1133   mspace_free behaves as free, but operates within
       
  1134   the given space.
       
  1135 
       
  1136   If compiled with FOOTERS==1, mspace_free is not actually needed.
       
  1137   free may be called instead of mspace_free because freed chunks from
       
  1138   any space are handled by their originating spaces.
       
  1139 */
       
  1140 void mspace_free(mspace msp, void* mem);
       
  1141 
       
  1142 /*
       
  1143   mspace_realloc behaves as realloc, but operates within
       
  1144   the given space.
       
  1145 
       
  1146   If compiled with FOOTERS==1, mspace_realloc is not actually
       
  1147   needed.  realloc may be called instead of mspace_realloc because
       
  1148   realloced chunks from any space are handled by their originating
       
  1149   spaces.
       
  1150 */
       
  1151 void* mspace_realloc(mspace msp, void* mem, size_t newsize);
       
  1152 
       
  1153 /*
       
  1154   mspace_calloc behaves as calloc, but operates within
       
  1155   the given space.
       
  1156 */
       
  1157 void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size);
       
  1158 
       
  1159 /*
       
  1160   mspace_memalign behaves as memalign, but operates within
       
  1161   the given space.
       
  1162 */
       
  1163 void* mspace_memalign(mspace msp, size_t alignment, size_t bytes);
       
  1164 
       
  1165 /*
       
  1166   mspace_independent_calloc behaves as independent_calloc, but
       
  1167   operates within the given space.
       
  1168 */
       
  1169 void** mspace_independent_calloc(mspace msp, size_t n_elements,
       
  1170                                  size_t elem_size, void* chunks[]);
       
  1171 
       
  1172 /*
       
  1173   mspace_independent_comalloc behaves as independent_comalloc, but
       
  1174   operates within the given space.
       
  1175 */
       
  1176 void** mspace_independent_comalloc(mspace msp, size_t n_elements,
       
  1177                                    size_t sizes[], void* chunks[]);
       
  1178 
       
  1179 /*
       
  1180   mspace_footprint() returns the number of bytes obtained from the
       
  1181   system for this space.
       
  1182 */
       
  1183 size_t mspace_footprint(mspace msp);
       
  1184 
       
  1185 /*
       
  1186   mspace_max_footprint() returns the peak number of bytes obtained from the
       
  1187   system for this space.
       
  1188 */
       
  1189 size_t mspace_max_footprint(mspace msp);
       
  1190 
       
  1191 
       
  1192 #if !NO_MALLINFO
       
  1193 /*
       
  1194   mspace_mallinfo behaves as mallinfo, but reports properties of
       
  1195   the given space.
       
  1196 */
       
  1197 struct mallinfo mspace_mallinfo(mspace msp);
       
  1198 #endif /* NO_MALLINFO */
       
  1199 
       
  1200 /*
       
  1201   mspace_malloc_stats behaves as malloc_stats, but reports
       
  1202   properties of the given space.
       
  1203 */
       
  1204 void mspace_malloc_stats(mspace msp);
       
  1205 
       
  1206 /*
       
  1207   mspace_trim behaves as malloc_trim, but
       
  1208   operates within the given space.
       
  1209 */
       
  1210 int mspace_trim(mspace msp, size_t pad);
       
  1211 
       
  1212 /*
       
  1213   An alias for mallopt.
       
  1214 */
       
  1215 int mspace_mallopt(int, int);
       
  1216 
       
  1217 #endif /* MSPACES */
       
  1218 
       
  1219 #ifdef __cplusplus
       
  1220 };  /* end of extern "C" */
       
  1221 #endif /* __cplusplus */
       
  1222 
       
  1223 /*
       
  1224   ========================================================================
       
  1225   To make a fully customizable malloc.h header file, cut everything
       
  1226   above this line, put into file malloc.h, edit to suit, and #include it
       
  1227   on the next line, as well as in programs that use this malloc.
       
  1228   ========================================================================
       
  1229 */
       
  1230 
       
  1231 /* #include "malloc.h" */
       
  1232 
       
  1233 /*------------------------------ internal #includes ---------------------- */
       
  1234 
       
  1235 #ifdef WIN32
       
  1236 #pragma warning( disable : 4146 ) /* no "unsigned" warnings */
       
  1237 #endif /* WIN32 */
       
  1238 
       
  1239 #include <stdio.h>       /* for printing in malloc_stats */
       
  1240 
       
  1241 #ifndef LACKS_ERRNO_H
       
  1242 #include <errno.h>       /* for MALLOC_FAILURE_ACTION */
       
  1243 #endif /* LACKS_ERRNO_H */
       
  1244 #if FOOTERS
       
  1245 #include <time.h>        /* for magic initialization */
       
  1246 #endif /* FOOTERS */
       
  1247 #ifndef LACKS_STDLIB_H
       
  1248 #include <stdlib.h>      /* for abort() */
       
  1249 #endif /* LACKS_STDLIB_H */
       
  1250 #ifdef DEBUG
       
  1251 #if ABORT_ON_ASSERT_FAILURE
       
  1252 #define assert(x) if(!(x)) ABORT
       
  1253 #else /* ABORT_ON_ASSERT_FAILURE */
       
  1254 #include <assert.h>
       
  1255 #endif /* ABORT_ON_ASSERT_FAILURE */
       
  1256 #else  /* DEBUG */
       
  1257 #define assert(x)
       
  1258 #endif /* DEBUG */
       
  1259 #ifndef LACKS_STRING_H
       
  1260 #include <string.h>      /* for memset etc */
       
  1261 #endif  /* LACKS_STRING_H */
       
  1262 #if USE_BUILTIN_FFS
       
  1263 #ifndef LACKS_STRINGS_H
       
  1264 #include <strings.h>     /* for ffs */
       
  1265 #endif /* LACKS_STRINGS_H */
       
  1266 #endif /* USE_BUILTIN_FFS */
       
  1267 #if HAVE_MMAP
       
  1268 #ifndef LACKS_SYS_MMAN_H
       
  1269 #include <sys/mman.h>    /* for mmap */
       
  1270 #endif /* LACKS_SYS_MMAN_H */
       
  1271 #ifndef LACKS_FCNTL_H
       
  1272 #include <fcntl.h>
       
  1273 #endif /* LACKS_FCNTL_H */
       
  1274 #endif /* HAVE_MMAP */
       
  1275 #if HAVE_MORECORE
       
  1276 #ifndef LACKS_UNISTD_H
       
  1277 #include <unistd.h>     /* for sbrk */
       
  1278 #else /* LACKS_UNISTD_H */
       
  1279 #if !defined(__FreeBSD__) && !defined(__OpenBSD__) && !defined(__NetBSD__)
       
  1280 extern void*     sbrk(ptrdiff_t);
       
  1281 #endif /* FreeBSD etc */
       
  1282 #endif /* LACKS_UNISTD_H */
       
  1283 #endif /* HAVE_MMAP */
       
  1284 
       
  1285 /* Declarations for locking */
       
  1286 #if USE_LOCKS
       
  1287 #ifndef WIN32
       
  1288 #include <pthread.h>
       
  1289 #if defined (__SVR4) && defined (__sun)  /* solaris */
       
  1290 #include <thread.h>
       
  1291 #endif /* solaris */
       
  1292 #else
       
  1293 #ifndef _M_AMD64
       
  1294 /* These are already defined on AMD64 builds */
       
  1295 #ifdef __cplusplus
       
  1296 extern "C" {
       
  1297 #endif /* __cplusplus */
       
  1298 LONG __cdecl _InterlockedCompareExchange(LPLONG volatile Dest, LONG Exchange, LONG Comp);
       
  1299 LONG __cdecl _InterlockedExchange(LPLONG volatile Target, LONG Value);
       
  1300 #ifdef __cplusplus
       
  1301 }
       
  1302 #endif /* __cplusplus */
       
  1303 #endif /* _M_AMD64 */
       
  1304 #pragma intrinsic (_InterlockedCompareExchange)
       
  1305 #pragma intrinsic (_InterlockedExchange)
       
  1306 #define interlockedcompareexchange _InterlockedCompareExchange
       
  1307 #define interlockedexchange _InterlockedExchange
       
  1308 #endif /* Win32 */
       
  1309 #endif /* USE_LOCKS */
       
  1310 
       
  1311 /* Declarations for bit scanning on win32 */
       
  1312 #if defined(_MSC_VER) && _MSC_VER>=1300
       
  1313 #ifndef BitScanForward	/* Try to avoid pulling in WinNT.h */
       
  1314 #ifdef __cplusplus
       
  1315 extern "C" {
       
  1316 #endif /* __cplusplus */
       
  1317 unsigned char _BitScanForward(unsigned long *index, unsigned long mask);
       
  1318 unsigned char _BitScanReverse(unsigned long *index, unsigned long mask);
       
  1319 #ifdef __cplusplus
       
  1320 }
       
  1321 #endif /* __cplusplus */
       
  1322 
       
  1323 #define BitScanForward _BitScanForward
       
  1324 #define BitScanReverse _BitScanReverse
       
  1325 #pragma intrinsic(_BitScanForward)
       
  1326 #pragma intrinsic(_BitScanReverse)
       
  1327 #endif /* BitScanForward */
       
  1328 #endif /* defined(_MSC_VER) && _MSC_VER>=1300 */
       
  1329 
       
  1330 #ifndef WIN32
       
  1331 #ifndef malloc_getpagesize
       
  1332 #  ifdef _SC_PAGESIZE         /* some SVR4 systems omit an underscore */
       
  1333 #    ifndef _SC_PAGE_SIZE
       
  1334 #      define _SC_PAGE_SIZE _SC_PAGESIZE
       
  1335 #    endif
       
  1336 #  endif
       
  1337 #  ifdef _SC_PAGE_SIZE
       
  1338 #    define malloc_getpagesize sysconf(_SC_PAGE_SIZE)
       
  1339 #  else
       
  1340 #    if defined(BSD) || defined(DGUX) || defined(HAVE_GETPAGESIZE)
       
  1341        extern size_t getpagesize();
       
  1342 #      define malloc_getpagesize getpagesize()
       
  1343 #    else
       
  1344 #      ifdef WIN32 /* use supplied emulation of getpagesize */
       
  1345 #        define malloc_getpagesize getpagesize()
       
  1346 #      else
       
  1347 #        ifndef LACKS_SYS_PARAM_H
       
  1348 #          include <sys/param.h>
       
  1349 #        endif
       
  1350 #        ifdef EXEC_PAGESIZE
       
  1351 #          define malloc_getpagesize EXEC_PAGESIZE
       
  1352 #        else
       
  1353 #          ifdef NBPG
       
  1354 #            ifndef CLSIZE
       
  1355 #              define malloc_getpagesize NBPG
       
  1356 #            else
       
  1357 #              define malloc_getpagesize (NBPG * CLSIZE)
       
  1358 #            endif
       
  1359 #          else
       
  1360 #            ifdef NBPC
       
  1361 #              define malloc_getpagesize NBPC
       
  1362 #            else
       
  1363 #              ifdef PAGESIZE
       
  1364 #                define malloc_getpagesize PAGESIZE
       
  1365 #              else /* just guess */
       
  1366 #                define malloc_getpagesize ((size_t)4096U)
       
  1367 #              endif
       
  1368 #            endif
       
  1369 #          endif
       
  1370 #        endif
       
  1371 #      endif
       
  1372 #    endif
       
  1373 #  endif
       
  1374 #endif
       
  1375 #endif
       
  1376 
       
  1377 
       
  1378 
       
  1379 /* ------------------- size_t and alignment properties -------------------- */
       
  1380 
       
  1381 /* The byte and bit size of a size_t */
       
  1382 #define SIZE_T_SIZE         (sizeof(size_t))
       
  1383 #define SIZE_T_BITSIZE      (sizeof(size_t) << 3)
       
  1384 
       
  1385 /* Some constants coerced to size_t */
       
  1386 /* Annoying but necessary to avoid errors on some platforms */
       
  1387 #define SIZE_T_ZERO         ((size_t)0)
       
  1388 #define SIZE_T_ONE          ((size_t)1)
       
  1389 #define SIZE_T_TWO          ((size_t)2)
       
  1390 #define SIZE_T_FOUR         ((size_t)4)
       
  1391 #define TWO_SIZE_T_SIZES    (SIZE_T_SIZE<<1)
       
  1392 #define FOUR_SIZE_T_SIZES   (SIZE_T_SIZE<<2)
       
  1393 #define SIX_SIZE_T_SIZES    (FOUR_SIZE_T_SIZES+TWO_SIZE_T_SIZES)
       
  1394 #define HALF_MAX_SIZE_T     (MAX_SIZE_T / 2U)
       
  1395 
       
  1396 /* The bit mask value corresponding to MALLOC_ALIGNMENT */
       
  1397 #define CHUNK_ALIGN_MASK    (MALLOC_ALIGNMENT - SIZE_T_ONE)
       
  1398 
       
  1399 /* True if address a has acceptable alignment */
       
  1400 #define is_aligned(A)       (((size_t)((A)) & (CHUNK_ALIGN_MASK)) == 0)
       
  1401 
       
  1402 /* the number of bytes to offset an address to align it */
       
  1403 #define align_offset(A)\
       
  1404  ((((size_t)(A) & CHUNK_ALIGN_MASK) == 0)? 0 :\
       
  1405   ((MALLOC_ALIGNMENT - ((size_t)(A) & CHUNK_ALIGN_MASK)) & CHUNK_ALIGN_MASK))
       
  1406 
       
  1407 /* -------------------------- MMAP preliminaries ------------------------- */
       
  1408 
       
  1409 /*
       
  1410    If HAVE_MORECORE or HAVE_MMAP are false, we just define calls and
       
  1411    checks to fail so compiler optimizer can delete code rather than
       
  1412    using so many "#if"s.
       
  1413 */
       
  1414 
       
  1415 
       
  1416 /* MORECORE and MMAP must return MFAIL on failure */
       
  1417 #define MFAIL                ((void*)(MAX_SIZE_T))
       
  1418 #define CMFAIL               ((char*)(MFAIL)) /* defined for convenience */
       
  1419 
       
  1420 #if !HAVE_MMAP
       
  1421 #define IS_MMAPPED_BIT       (SIZE_T_ZERO)
       
  1422 #define USE_MMAP_BIT         (SIZE_T_ZERO)
       
  1423 #define CALL_MMAP(s)         MFAIL
       
  1424 #define CALL_MUNMAP(a, s)    (-1)
       
  1425 #define DIRECT_MMAP(s)       MFAIL
       
  1426 
       
  1427 #else /* HAVE_MMAP */
       
  1428 #define IS_MMAPPED_BIT       (SIZE_T_ONE)
       
  1429 #define USE_MMAP_BIT         (SIZE_T_ONE)
       
  1430 
       
  1431 #ifndef WIN32
       
  1432 #define CALL_MUNMAP(a, s)    munmap((a), (s))
       
  1433 #define MMAP_PROT            (PROT_READ|PROT_WRITE)
       
  1434 #if !defined(MAP_ANONYMOUS) && defined(MAP_ANON)
       
  1435 #define MAP_ANONYMOUS        MAP_ANON
       
  1436 #endif /* MAP_ANON */
       
  1437 #ifdef MAP_ANONYMOUS
       
  1438 #define MMAP_FLAGS           (MAP_PRIVATE|MAP_ANONYMOUS)
       
  1439 #define CALL_MMAP(s)         mmap(0, (s), MMAP_PROT, MMAP_FLAGS, -1, 0)
       
  1440 #else /* MAP_ANONYMOUS */
       
  1441 /*
       
  1442    Nearly all versions of mmap support MAP_ANONYMOUS, so the following
       
  1443    is unlikely to be needed, but is supplied just in case.
       
  1444 */
       
  1445 #define MMAP_FLAGS           (MAP_PRIVATE)
       
  1446 static int dev_zero_fd = -1; /* Cached file descriptor for /dev/zero. */
       
  1447 #define CALL_MMAP(s) ((dev_zero_fd < 0) ? \
       
  1448            (dev_zero_fd = open("/dev/zero", O_RDWR), \
       
  1449             mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0)) : \
       
  1450             mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0))
       
  1451 #endif /* MAP_ANONYMOUS */
       
  1452 
       
  1453 #define DIRECT_MMAP(s)       CALL_MMAP(s)
       
  1454 #else /* WIN32 */
       
  1455 
       
  1456 /* Win32 MMAP via VirtualAlloc */
       
  1457 static FORCEINLINE void* win32mmap(size_t size) {
       
  1458   void* ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
       
  1459   return (ptr != 0)? ptr: MFAIL;
       
  1460 }
       
  1461 
       
  1462 /* For direct MMAP, use MEM_TOP_DOWN to minimize interference */
       
  1463 static FORCEINLINE void* win32direct_mmap(size_t size) {
       
  1464   void* ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT|MEM_TOP_DOWN,
       
  1465                            PAGE_READWRITE);
       
  1466   return (ptr != 0)? ptr: MFAIL;
       
  1467 }
       
  1468 
       
  1469 /* This function supports releasing coalesed segments */
       
  1470 static FORCEINLINE int win32munmap(void* ptr, size_t size) {
       
  1471   MEMORY_BASIC_INFORMATION minfo;
       
  1472   char* cptr = (char*)ptr;
       
  1473   while (size) {
       
  1474     if (VirtualQuery(cptr, &minfo, sizeof(minfo)) == 0)
       
  1475       return -1;
       
  1476     if (minfo.BaseAddress != cptr || minfo.AllocationBase != cptr ||
       
  1477         minfo.State != MEM_COMMIT || minfo.RegionSize > size)
       
  1478       return -1;
       
  1479     if (VirtualFree(cptr, 0, MEM_RELEASE) == 0)
       
  1480       return -1;
       
  1481     cptr += minfo.RegionSize;
       
  1482     size -= minfo.RegionSize;
       
  1483   }
       
  1484   return 0;
       
  1485 }
       
  1486 
       
  1487 #define CALL_MMAP(s)         win32mmap(s)
       
  1488 #define CALL_MUNMAP(a, s)    win32munmap((a), (s))
       
  1489 #define DIRECT_MMAP(s)       win32direct_mmap(s)
       
  1490 #endif /* WIN32 */
       
  1491 #endif /* HAVE_MMAP */
       
  1492 
       
  1493 #if HAVE_MMAP && HAVE_MREMAP
       
  1494 #define CALL_MREMAP(addr, osz, nsz, mv) mremap((addr), (osz), (nsz), (mv))
       
  1495 #else  /* HAVE_MMAP && HAVE_MREMAP */
       
  1496 #define CALL_MREMAP(addr, osz, nsz, mv) ((void)(addr),(void)(osz), \
       
  1497                                          (void)(nsz), (void)(mv),MFAIL)
       
  1498 #endif /* HAVE_MMAP && HAVE_MREMAP */
       
  1499 
       
  1500 #if HAVE_MORECORE
       
  1501 #define CALL_MORECORE(S)     MORECORE(S)
       
  1502 #else  /* HAVE_MORECORE */
       
  1503 #define CALL_MORECORE(S)     MFAIL
       
  1504 #endif /* HAVE_MORECORE */
       
  1505 
       
  1506 /* mstate bit set if continguous morecore disabled or failed */
       
  1507 #define USE_NONCONTIGUOUS_BIT (4U)
       
  1508 
       
  1509 /* segment bit set in create_mspace_with_base */
       
  1510 #define EXTERN_BIT            (8U)
       
  1511 
       
  1512 
       
  1513 /* --------------------------- Lock preliminaries ------------------------ */
       
  1514 
       
  1515 /*
       
  1516   When locks are defined, there are up to two global locks:
       
  1517 
       
  1518   * If HAVE_MORECORE, morecore_mutex protects sequences of calls to
       
  1519     MORECORE.  In many cases sys_alloc requires two calls, that should
       
  1520     not be interleaved with calls by other threads.  This does not
       
  1521     protect against direct calls to MORECORE by other threads not
       
  1522     using this lock, so there is still code to cope the best we can on
       
  1523     interference.
       
  1524 
       
  1525   * magic_init_mutex ensures that mparams.magic and other
       
  1526     unique mparams values are initialized only once.
       
  1527 
       
  1528    To enable use in layered extensions, locks are reentrant.
       
  1529 
       
  1530    Because lock-protected regions generally have bounded times, we use
       
  1531    the supplied simple spinlocks in the custom versions for x86.
       
  1532 
       
  1533    If USE_LOCKS is > 1, the definitions of lock routines here are
       
  1534    bypassed, in which case you will need to define at least
       
  1535    INITIAL_LOCK, ACQUIRE_LOCK, RELEASE_LOCK, and
       
  1536    NULL_LOCK_INITIALIZER, and possibly TRY_LOCK and IS_LOCKED
       
  1537    (The latter two are not used in this malloc, but are
       
  1538    commonly needed in extensions.)
       
  1539 */
       
  1540 
       
  1541 #if USE_LOCKS == 1
       
  1542 
       
  1543 #if USE_SPIN_LOCKS
       
  1544 #ifndef WIN32
       
  1545 /* Custom pthread-style spin locks on x86 and x64 for gcc */
       
  1546 struct pthread_mlock_t
       
  1547 {
       
  1548   volatile pthread_t threadid;
       
  1549   volatile unsigned int c;
       
  1550   volatile unsigned int l;
       
  1551 };
       
  1552 #define MLOCK_T struct pthread_mlock_t
       
  1553 #define CURRENT_THREAD        pthread_self()
       
  1554 #define SPINS_PER_YIELD       63
       
  1555 static FORCEINLINE int pthread_acquire_lock (MLOCK_T *sl) {
       
  1556   if(CURRENT_THREAD==sl->threadid)
       
  1557     ++sl->c;
       
  1558   else {
       
  1559     int spins = 0;
       
  1560     for (;;) {
       
  1561       int ret;
       
  1562       __asm__ __volatile__ ("lock cmpxchgl %2,(%1)" : "=a" (ret) : "r" (&sl->l), "r" (1), "a" (0));
       
  1563       if(!ret) {
       
  1564         assert(!sl->threadid);
       
  1565         sl->threadid=CURRENT_THREAD;
       
  1566         sl->c=1;
       
  1567         break;
       
  1568       }
       
  1569       if ((++spins & SPINS_PER_YIELD) == 0) {
       
  1570 #if defined (__SVR4) && defined (__sun) /* solaris */
       
  1571         thr_yield();
       
  1572 #else
       
  1573 #ifdef linux
       
  1574         sched_yield();
       
  1575 #else  /* no-op yield on unknown systems */
       
  1576         ;
       
  1577 #endif /* linux */
       
  1578 #endif /* solaris */
       
  1579       }
       
  1580     }
       
  1581   }
       
  1582 
       
  1583   return 0;
       
  1584 }
       
  1585 
       
  1586 static FORCEINLINE void pthread_release_lock (MLOCK_T *sl) {
       
  1587   int ret;
       
  1588   assert(CURRENT_THREAD==sl->threadid);
       
  1589   if (!--sl->c) {
       
  1590     sl->threadid=0;
       
  1591     __asm__ __volatile__ ("xchgl %2,(%1)" : "=r" (ret) : "r" (&sl->l), "0" (0));
       
  1592   }
       
  1593 }
       
  1594 
       
  1595 static FORCEINLINE int pthread_try_lock (MLOCK_T *sl) {
       
  1596   int ret;
       
  1597   __asm__ __volatile__ ("lock cmpxchgl %2,(%1)" : "=a" (ret) : "r" (&sl->l), "r" (1), "a" (0));
       
  1598   if(!ret){
       
  1599     assert(!sl->threadid);
       
  1600     sl->threadid=CURRENT_THREAD;
       
  1601     sl->c=1;
       
  1602     return 1;
       
  1603   }
       
  1604   return 0;
       
  1605 }
       
  1606 
       
  1607 #define INITIAL_LOCK(sl)      (memset((sl), 0, sizeof(MLOCK_T)), 0)
       
  1608 #define ACQUIRE_LOCK(sl)      pthread_acquire_lock(sl)
       
  1609 #define RELEASE_LOCK(sl)      pthread_release_lock(sl)
       
  1610 #define TRY_LOCK(sl)          pthread_try_lock(sl)
       
  1611 #define IS_LOCKED(sl)         ((sl)->l)
       
  1612 
       
  1613 static MLOCK_T magic_init_mutex = {0, 0, 0 };
       
  1614 #if HAVE_MORECORE
       
  1615 static MLOCK_T morecore_mutex = {0, 0, 0 };
       
  1616 #endif /* HAVE_MORECORE */
       
  1617 
       
  1618 #else /* WIN32 */
       
  1619 /* Custom win32-style spin locks on x86 and x64 for MSC */
       
  1620 struct win32_mlock_t
       
  1621 {
       
  1622   volatile long threadid;
       
  1623   volatile unsigned int c;
       
  1624   long l;
       
  1625 };
       
  1626 #define MLOCK_T struct win32_mlock_t
       
  1627 #define CURRENT_THREAD        GetCurrentThreadId()
       
  1628 #define SPINS_PER_YIELD    63
       
  1629 static FORCEINLINE int win32_acquire_lock (MLOCK_T *sl) {
       
  1630   long mythreadid=CURRENT_THREAD;
       
  1631   if(mythreadid==sl->threadid)
       
  1632     ++sl->c;
       
  1633   else {
       
  1634     int spins = 0;
       
  1635     for (;;) {
       
  1636       if (!interlockedexchange(&sl->l, 1)) {
       
  1637         assert(!sl->threadid);
       
  1638         sl->threadid=mythreadid;
       
  1639         sl->c=1;
       
  1640         break;
       
  1641       }
       
  1642       if ((++spins & SPINS_PER_YIELD) == 0)
       
  1643         SleepEx(0, FALSE);
       
  1644     }
       
  1645   }
       
  1646   return 0;
       
  1647 }
       
  1648 
       
  1649 static FORCEINLINE void win32_release_lock (MLOCK_T *sl) {
       
  1650   assert(CURRENT_THREAD==sl->threadid);
       
  1651   if (!--sl->c) {
       
  1652     sl->threadid=0;
       
  1653     interlockedexchange (&sl->l, 0);
       
  1654   }
       
  1655 }
       
  1656 
       
  1657 static FORCEINLINE int win32_try_lock (MLOCK_T *sl) {
       
  1658   if (!interlockedexchange(&sl->l, 1)){
       
  1659     assert(!sl->threadid);
       
  1660     sl->threadid=CURRENT_THREAD;
       
  1661     sl->c=1;
       
  1662     return 1;
       
  1663   }
       
  1664   return 0;
       
  1665 }
       
  1666 
       
  1667 #define INITIAL_LOCK(sl)      (memset(sl, 0, sizeof(MLOCK_T)), 0)
       
  1668 #define ACQUIRE_LOCK(sl)      win32_acquire_lock(sl)
       
  1669 #define RELEASE_LOCK(sl)      win32_release_lock(sl)
       
  1670 #define TRY_LOCK(sl)          win32_try_lock(sl)
       
  1671 #define IS_LOCKED(sl)         ((sl)->l)
       
  1672 
       
  1673 static MLOCK_T magic_init_mutex = {0, 0 };
       
  1674 #if HAVE_MORECORE
       
  1675 static MLOCK_T morecore_mutex = {0, 0 };
       
  1676 #endif /* HAVE_MORECORE */
       
  1677 
       
  1678 #endif /* WIN32 */
       
  1679 #else /* USE_SPIN_LOCKS */
       
  1680 
       
  1681 #ifndef WIN32
       
  1682 /* pthreads-based locks */
       
  1683 struct pthread_mlock_t
       
  1684 {
       
  1685   volatile unsigned int c;
       
  1686   pthread_mutex_t l;
       
  1687 };
       
  1688 #define MLOCK_T struct pthread_mlock_t
       
  1689 #define CURRENT_THREAD        pthread_self()
       
  1690 static FORCEINLINE int pthread_acquire_lock (MLOCK_T *sl) {
       
  1691   if(!pthread_mutex_lock(&(sl)->l)){
       
  1692     sl->c++;
       
  1693     return 0;
       
  1694   }
       
  1695   return 1;
       
  1696 }
       
  1697 
       
  1698 static FORCEINLINE void pthread_release_lock (MLOCK_T *sl) {
       
  1699   --sl->c;
       
  1700   pthread_mutex_unlock(&(sl)->l);
       
  1701 }
       
  1702 
       
  1703 static FORCEINLINE int pthread_try_lock (MLOCK_T *sl) {
       
  1704   if(!pthread_mutex_trylock(&(sl)->l)){
       
  1705     sl->c++;
       
  1706     return 1;
       
  1707   }
       
  1708   return 0;
       
  1709 }
       
  1710 
       
  1711 static FORCEINLINE int pthread_init_lock (MLOCK_T *sl) {
       
  1712   pthread_mutexattr_t attr;
       
  1713   sl->c=0;
       
  1714   if(pthread_mutexattr_init(&attr)) return 1;
       
  1715   if(pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE)) return 1;
       
  1716   if(pthread_mutex_init(&sl->l, &attr)) return 1;
       
  1717   pthread_mutexattr_destroy(&attr);
       
  1718   return 0;
       
  1719 }
       
  1720 
       
  1721 static FORCEINLINE int pthread_islocked (MLOCK_T *sl) {
       
  1722   if(!pthread_try_lock(sl)){
       
  1723     int ret = (sl->c != 0);
       
  1724     pthread_mutex_unlock(sl);
       
  1725     return ret;
       
  1726   }
       
  1727   return 0;
       
  1728 }
       
  1729 
       
  1730 #define INITIAL_LOCK(sl)      pthread_init_lock(sl)
       
  1731 #define ACQUIRE_LOCK(sl)      pthread_acquire_lock(sl)
       
  1732 #define RELEASE_LOCK(sl)      pthread_release_lock(sl)
       
  1733 #define TRY_LOCK(sl)          pthread_try_lock(sl)
       
  1734 #define IS_LOCKED(sl)         pthread_islocked(sl)
       
  1735 
       
  1736 static MLOCK_T magic_init_mutex = {0, PTHREAD_MUTEX_INITIALIZER };
       
  1737 #if HAVE_MORECORE
       
  1738 static MLOCK_T morecore_mutex = {0, PTHREAD_MUTEX_INITIALIZER };
       
  1739 #endif /* HAVE_MORECORE */
       
  1740 
       
  1741 #else /* WIN32 */
       
  1742 /* Win32 critical sections */
       
  1743 #define MLOCK_T         CRITICAL_SECTION
       
  1744 #define CURRENT_THREAD  GetCurrentThreadId()
       
  1745 #define INITIAL_LOCK(s) (!InitializeCriticalSectionAndSpinCount((s), 4000)
       
  1746 #define ACQUIRE_LOCK(s) ( (!((s))->DebugInfo ? INITIAL_LOCK((s)) : 0), !EnterCriticalSection((s)), 0)
       
  1747 #define RELEASE_LOCK(s) ( LeaveCriticalSection((s)), 0 )
       
  1748 #define TRY_LOCK(s)     ( TryEnterCriticalSection((s)) )
       
  1749 #define IS_LOCKED(s)    ( (s)->LockCount >= 0 )
       
  1750 #define NULL_LOCK_INITIALIZER
       
  1751 static MLOCK_T magic_init_mutex;
       
  1752 #if HAVE_MORECORE
       
  1753 static MLOCK_T morecore_mutex;
       
  1754 #endif /* HAVE_MORECORE */
       
  1755 #endif /* WIN32 */
       
  1756 #endif /* USE_SPIN_LOCKS */
       
  1757 #endif /* USE_LOCKS == 1 */
       
  1758 
       
  1759 /* -----------------------  User-defined locks ------------------------ */
       
  1760 
       
  1761 #if USE_LOCKS > 1
       
  1762 /* Define your own lock implementation here */
       
  1763 /* #define INITIAL_LOCK(sl)  ... */
       
  1764 /* #define ACQUIRE_LOCK(sl)  ... */
       
  1765 /* #define RELEASE_LOCK(sl)  ... */
       
  1766 /* #define TRY_LOCK(sl) ... */
       
  1767 /* #define IS_LOCKED(sl) ... */
       
  1768 /* #define NULL_LOCK_INITIALIZER ... */
       
  1769 
       
  1770 static MLOCK_T magic_init_mutex = NULL_LOCK_INITIALIZER;
       
  1771 #if HAVE_MORECORE
       
  1772 static MLOCK_T morecore_mutex = NULL_LOCK_INITIALIZER;
       
  1773 #endif /* HAVE_MORECORE */
       
  1774 #endif /* USE_LOCKS > 1 */
       
  1775 
       
  1776 /* -----------------------  Lock-based state ------------------------ */
       
  1777 
       
  1778 
       
  1779 #if USE_LOCKS
       
  1780 #define USE_LOCK_BIT               (2U)
       
  1781 #else  /* USE_LOCKS */
       
  1782 #define USE_LOCK_BIT               (0U)
       
  1783 #define INITIAL_LOCK(l)
       
  1784 #endif /* USE_LOCKS */
       
  1785 
       
  1786 #if USE_LOCKS && HAVE_MORECORE
       
  1787 #define ACQUIRE_MORECORE_LOCK()    ACQUIRE_LOCK(&morecore_mutex);
       
  1788 #define RELEASE_MORECORE_LOCK()    RELEASE_LOCK(&morecore_mutex);
       
  1789 #else /* USE_LOCKS && HAVE_MORECORE */
       
  1790 #define ACQUIRE_MORECORE_LOCK()
       
  1791 #define RELEASE_MORECORE_LOCK()
       
  1792 #endif /* USE_LOCKS && HAVE_MORECORE */
       
  1793 
       
  1794 #if USE_LOCKS
       
  1795 #define ACQUIRE_MAGIC_INIT_LOCK()  ACQUIRE_LOCK(&magic_init_mutex);
       
  1796 #define RELEASE_MAGIC_INIT_LOCK()  RELEASE_LOCK(&magic_init_mutex);
       
  1797 #else  /* USE_LOCKS */
       
  1798 #define ACQUIRE_MAGIC_INIT_LOCK()
       
  1799 #define RELEASE_MAGIC_INIT_LOCK()
       
  1800 #endif /* USE_LOCKS */
       
  1801 
       
  1802 
       
  1803 /* -----------------------  Chunk representations ------------------------ */
       
  1804 
       
  1805 /*
       
  1806   (The following includes lightly edited explanations by Colin Plumb.)
       
  1807 
       
  1808   The malloc_chunk declaration below is misleading (but accurate and
       
  1809   necessary).  It declares a "view" into memory allowing access to
       
  1810   necessary fields at known offsets from a given base.
       
  1811 
       
  1812   Chunks of memory are maintained using a `boundary tag' method as
       
  1813   originally described by Knuth.  (See the paper by Paul Wilson
       
  1814   ftp://ftp.cs.utexas.edu/pub/garbage/allocsrv.ps for a survey of such
       
  1815   techniques.)  Sizes of free chunks are stored both in the front of
       
  1816   each chunk and at the end.  This makes consolidating fragmented
       
  1817   chunks into bigger chunks fast.  The head fields also hold bits
       
  1818   representing whether chunks are free or in use.
       
  1819 
       
  1820   Here are some pictures to make it clearer.  They are "exploded" to
       
  1821   show that the state of a chunk can be thought of as extending from
       
  1822   the high 31 bits of the head field of its header through the
       
  1823   prev_foot and PINUSE_BIT bit of the following chunk header.
       
  1824 
       
  1825   A chunk that's in use looks like:
       
  1826 
       
  1827    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1828            | Size of previous chunk (if P = 1)                             |
       
  1829            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1830          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P|
       
  1831          | Size of this chunk                                         1| +-+
       
  1832    mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1833          |                                                               |
       
  1834          +-                                                             -+
       
  1835          |                                                               |
       
  1836          +-                                                             -+
       
  1837          |                                                               :
       
  1838          +-      size - sizeof(size_t) available payload bytes          -+
       
  1839          :                                                               |
       
  1840  chunk-> +-                                                             -+
       
  1841          |                                                               |
       
  1842          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1843        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|
       
  1844        | Size of next chunk (may or may not be in use)               | +-+
       
  1845  mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1846 
       
  1847     And if it's free, it looks like this:
       
  1848 
       
  1849    chunk-> +-                                                             -+
       
  1850            | User payload (must be in use, or we would have merged!)       |
       
  1851            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1852          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P|
       
  1853          | Size of this chunk                                         0| +-+
       
  1854    mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1855          | Next pointer                                                  |
       
  1856          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1857          | Prev pointer                                                  |
       
  1858          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1859          |                                                               :
       
  1860          +-      size - sizeof(struct chunk) unused bytes               -+
       
  1861          :                                                               |
       
  1862  chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1863          | Size of this chunk                                            |
       
  1864          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1865        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|
       
  1866        | Size of next chunk (must be in use, or we would have merged)| +-+
       
  1867  mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1868        |                                                               :
       
  1869        +- User payload                                                -+
       
  1870        :                                                               |
       
  1871        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  1872                                                                      |0|
       
  1873                                                                      +-+
       
  1874   Note that since we always merge adjacent free chunks, the chunks
       
  1875   adjacent to a free chunk must be in use.
       
  1876 
       
  1877   Given a pointer to a chunk (which can be derived trivially from the
       
  1878   payload pointer) we can, in O(1) time, find out whether the adjacent
       
  1879   chunks are free, and if so, unlink them from the lists that they
       
  1880   are on and merge them with the current chunk.
       
  1881 
       
  1882   Chunks always begin on even word boundaries, so the mem portion
       
  1883   (which is returned to the user) is also on an even word boundary, and
       
  1884   thus at least double-word aligned.
       
  1885 
       
  1886   The P (PINUSE_BIT) bit, stored in the unused low-order bit of the
       
  1887   chunk size (which is always a multiple of two words), is an in-use
       
  1888   bit for the *previous* chunk.  If that bit is *clear*, then the
       
  1889   word before the current chunk size contains the previous chunk
       
  1890   size, and can be used to find the front of the previous chunk.
       
  1891   The very first chunk allocated always has this bit set, preventing
       
  1892   access to non-existent (or non-owned) memory. If pinuse is set for
       
  1893   any given chunk, then you CANNOT determine the size of the
       
  1894   previous chunk, and might even get a memory addressing fault when
       
  1895   trying to do so.
       
  1896 
       
  1897   The C (CINUSE_BIT) bit, stored in the unused second-lowest bit of
       
  1898   the chunk size redundantly records whether the current chunk is
       
  1899   inuse. This redundancy enables usage checks within free and realloc,
       
  1900   and reduces indirection when freeing and consolidating chunks.
       
  1901 
       
  1902   Each freshly allocated chunk must have both cinuse and pinuse set.
       
  1903   That is, each allocated chunk borders either a previously allocated
       
  1904   and still in-use chunk, or the base of its memory arena. This is
       
  1905   ensured by making all allocations from the the `lowest' part of any
       
  1906   found chunk.  Further, no free chunk physically borders another one,
       
  1907   so each free chunk is known to be preceded and followed by either
       
  1908   inuse chunks or the ends of memory.
       
  1909 
       
  1910   Note that the `foot' of the current chunk is actually represented
       
  1911   as the prev_foot of the NEXT chunk. This makes it easier to
       
  1912   deal with alignments etc but can be very confusing when trying
       
  1913   to extend or adapt this code.
       
  1914 
       
  1915   The exceptions to all this are
       
  1916 
       
  1917      1. The special chunk `top' is the top-most available chunk (i.e.,
       
  1918         the one bordering the end of available memory). It is treated
       
  1919         specially.  Top is never included in any bin, is used only if
       
  1920         no other chunk is available, and is released back to the
       
  1921         system if it is very large (see M_TRIM_THRESHOLD).  In effect,
       
  1922         the top chunk is treated as larger (and thus less well
       
  1923         fitting) than any other available chunk.  The top chunk
       
  1924         doesn't update its trailing size field since there is no next
       
  1925         contiguous chunk that would have to index off it. However,
       
  1926         space is still allocated for it (TOP_FOOT_SIZE) to enable
       
  1927         separation or merging when space is extended.
       
  1928 
       
  1929      3. Chunks allocated via mmap, which have the lowest-order bit
       
  1930         (IS_MMAPPED_BIT) set in their prev_foot fields, and do not set
       
  1931         PINUSE_BIT in their head fields.  Because they are allocated
       
  1932         one-by-one, each must carry its own prev_foot field, which is
       
  1933         also used to hold the offset this chunk has within its mmapped
       
  1934         region, which is needed to preserve alignment. Each mmapped
       
  1935         chunk is trailed by the first two fields of a fake next-chunk
       
  1936         for sake of usage checks.
       
  1937 
       
  1938 */
       
  1939 
       
  1940 struct malloc_chunk {
       
  1941   size_t               prev_foot;  /* Size of previous chunk (if free).  */
       
  1942   size_t               head;       /* Size and inuse bits. */
       
  1943   struct malloc_chunk* fd;         /* double links -- used only if free. */
       
  1944   struct malloc_chunk* bk;
       
  1945 };
       
  1946 
       
  1947 typedef struct malloc_chunk  mchunk;
       
  1948 typedef struct malloc_chunk* mchunkptr;
       
  1949 typedef struct malloc_chunk* sbinptr;  /* The type of bins of chunks */
       
  1950 typedef unsigned int bindex_t;         /* Described below */
       
  1951 typedef unsigned int binmap_t;         /* Described below */
       
  1952 typedef unsigned int flag_t;           /* The type of various bit flag sets */
       
  1953 
       
  1954 /* ------------------- Chunks sizes and alignments ----------------------- */
       
  1955 
       
  1956 #define MCHUNK_SIZE         (sizeof(mchunk))
       
  1957 
       
  1958 #if FOOTERS
       
  1959 #define CHUNK_OVERHEAD      (TWO_SIZE_T_SIZES)
       
  1960 #else /* FOOTERS */
       
  1961 #define CHUNK_OVERHEAD      (SIZE_T_SIZE)
       
  1962 #endif /* FOOTERS */
       
  1963 
       
  1964 /* MMapped chunks need a second word of overhead ... */
       
  1965 #define MMAP_CHUNK_OVERHEAD (TWO_SIZE_T_SIZES)
       
  1966 /* ... and additional padding for fake next-chunk at foot */
       
  1967 #define MMAP_FOOT_PAD       (FOUR_SIZE_T_SIZES)
       
  1968 
       
  1969 /* The smallest size we can malloc is an aligned minimal chunk */
       
  1970 #define MIN_CHUNK_SIZE\
       
  1971   ((MCHUNK_SIZE + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK)
       
  1972 
       
  1973 /* conversion from malloc headers to user pointers, and back */
       
  1974 #define chunk2mem(p)        ((void*)((char*)(p)       + TWO_SIZE_T_SIZES))
       
  1975 #define mem2chunk(mem)      ((mchunkptr)((char*)(mem) - TWO_SIZE_T_SIZES))
       
  1976 /* chunk associated with aligned address A */
       
  1977 #define align_as_chunk(A)   (mchunkptr)((A) + align_offset(chunk2mem(A)))
       
  1978 
       
  1979 /* Bounds on request (not chunk) sizes. */
       
  1980 #define MAX_REQUEST         ((-MIN_CHUNK_SIZE) << 2)
       
  1981 #define MIN_REQUEST         (MIN_CHUNK_SIZE - CHUNK_OVERHEAD - SIZE_T_ONE)
       
  1982 
       
  1983 /* pad request bytes into a usable size */
       
  1984 #define pad_request(req) \
       
  1985    (((req) + CHUNK_OVERHEAD + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK)
       
  1986 
       
  1987 /* pad request, checking for minimum (but not maximum) */
       
  1988 #define request2size(req) \
       
  1989   (((req) < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(req))
       
  1990 
       
  1991 
       
  1992 /* ------------------ Operations on head and foot fields ----------------- */
       
  1993 
       
  1994 /*
       
  1995   The head field of a chunk is or'ed with PINUSE_BIT when previous
       
  1996   adjacent chunk in use, and or'ed with CINUSE_BIT if this chunk is in
       
  1997   use. If the chunk was obtained with mmap, the prev_foot field has
       
  1998   IS_MMAPPED_BIT set, otherwise holding the offset of the base of the
       
  1999   mmapped region to the base of the chunk.
       
  2000 
       
  2001   FLAG4_BIT is not used by this malloc, but might be useful in extensions.
       
  2002 */
       
  2003 
       
  2004 #define PINUSE_BIT          (SIZE_T_ONE)
       
  2005 #define CINUSE_BIT          (SIZE_T_TWO)
       
  2006 #define FLAG4_BIT           (SIZE_T_FOUR)
       
  2007 #define INUSE_BITS          (PINUSE_BIT|CINUSE_BIT)
       
  2008 #define FLAG_BITS           (PINUSE_BIT|CINUSE_BIT|FLAG4_BIT)
       
  2009 
       
  2010 /* Head value for fenceposts */
       
  2011 #define FENCEPOST_HEAD      (INUSE_BITS|SIZE_T_SIZE)
       
  2012 
       
  2013 /* extraction of fields from head words */
       
  2014 #define cinuse(p)           ((p)->head & CINUSE_BIT)
       
  2015 #define pinuse(p)           ((p)->head & PINUSE_BIT)
       
  2016 #define chunksize(p)        ((p)->head & ~(FLAG_BITS))
       
  2017 
       
  2018 #define clear_pinuse(p)     ((p)->head &= ~PINUSE_BIT)
       
  2019 #define clear_cinuse(p)     ((p)->head &= ~CINUSE_BIT)
       
  2020 
       
  2021 /* Treat space at ptr +/- offset as a chunk */
       
  2022 #define chunk_plus_offset(p, s)  ((mchunkptr)(((char*)(p)) + (s)))
       
  2023 #define chunk_minus_offset(p, s) ((mchunkptr)(((char*)(p)) - (s)))
       
  2024 
       
  2025 /* Ptr to next or previous physical malloc_chunk. */
       
  2026 #define next_chunk(p) ((mchunkptr)( ((char*)(p)) + ((p)->head & ~FLAG_BITS)))
       
  2027 #define prev_chunk(p) ((mchunkptr)( ((char*)(p)) - ((p)->prev_foot) ))
       
  2028 
       
  2029 /* extract next chunk's pinuse bit */
       
  2030 #define next_pinuse(p)  ((next_chunk(p)->head) & PINUSE_BIT)
       
  2031 
       
  2032 /* Get/set size at footer */
       
  2033 #define get_foot(p, s)  (((mchunkptr)((char*)(p) + (s)))->prev_foot)
       
  2034 #define set_foot(p, s)  (((mchunkptr)((char*)(p) + (s)))->prev_foot = (s))
       
  2035 
       
  2036 /* Set size, pinuse bit, and foot */
       
  2037 #define set_size_and_pinuse_of_free_chunk(p, s)\
       
  2038   ((p)->head = (s|PINUSE_BIT), set_foot(p, s))
       
  2039 
       
  2040 /* Set size, pinuse bit, foot, and clear next pinuse */
       
  2041 #define set_free_with_pinuse(p, s, n)\
       
  2042   (clear_pinuse(n), set_size_and_pinuse_of_free_chunk(p, s))
       
  2043 
       
  2044 #define is_mmapped(p)\
       
  2045   (!((p)->head & PINUSE_BIT) && ((p)->prev_foot & IS_MMAPPED_BIT))
       
  2046 
       
  2047 /* Get the internal overhead associated with chunk p */
       
  2048 #define overhead_for(p)\
       
  2049  (is_mmapped(p)? MMAP_CHUNK_OVERHEAD : CHUNK_OVERHEAD)
       
  2050 
       
  2051 /* Return true if malloced space is not necessarily cleared */
       
  2052 #if MMAP_CLEARS
       
  2053 #define calloc_must_clear(p) (!is_mmapped(p))
       
  2054 #else /* MMAP_CLEARS */
       
  2055 #define calloc_must_clear(p) (1)
       
  2056 #endif /* MMAP_CLEARS */
       
  2057 
       
  2058 /* ---------------------- Overlaid data structures ----------------------- */
       
  2059 
       
  2060 /*
       
  2061   When chunks are not in use, they are treated as nodes of either
       
  2062   lists or trees.
       
  2063 
       
  2064   "Small"  chunks are stored in circular doubly-linked lists, and look
       
  2065   like this:
       
  2066 
       
  2067     chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2068             |             Size of previous chunk                            |
       
  2069             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2070     `head:' |             Size of chunk, in bytes                         |P|
       
  2071       mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2072             |             Forward pointer to next chunk in list             |
       
  2073             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2074             |             Back pointer to previous chunk in list            |
       
  2075             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2076             |             Unused space (may be 0 bytes long)                .
       
  2077             .                                                               .
       
  2078             .                                                               |
       
  2079 nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2080     `foot:' |             Size of chunk, in bytes                           |
       
  2081             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2082 
       
  2083   Larger chunks are kept in a form of bitwise digital trees (aka
       
  2084   tries) keyed on chunksizes.  Because malloc_tree_chunks are only for
       
  2085   free chunks greater than 256 bytes, their size doesn't impose any
       
  2086   constraints on user chunk sizes.  Each node looks like:
       
  2087 
       
  2088     chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2089             |             Size of previous chunk                            |
       
  2090             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2091     `head:' |             Size of chunk, in bytes                         |P|
       
  2092       mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2093             |             Forward pointer to next chunk of same size        |
       
  2094             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2095             |             Back pointer to previous chunk of same size       |
       
  2096             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2097             |             Pointer to left child (child[0])                  |
       
  2098             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2099             |             Pointer to right child (child[1])                 |
       
  2100             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2101             |             Pointer to parent                                 |
       
  2102             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2103             |             bin index of this chunk                           |
       
  2104             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2105             |             Unused space                                      .
       
  2106             .                                                               |
       
  2107 nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2108     `foot:' |             Size of chunk, in bytes                           |
       
  2109             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       
  2110 
       
  2111   Each tree holding treenodes is a tree of unique chunk sizes.  Chunks
       
  2112   of the same size are arranged in a circularly-linked list, with only
       
  2113   the oldest chunk (the next to be used, in our FIFO ordering)
       
  2114   actually in the tree.  (Tree members are distinguished by a non-null
       
  2115   parent pointer.)  If a chunk with the same size an an existing node
       
  2116   is inserted, it is linked off the existing node using pointers that
       
  2117   work in the same way as fd/bk pointers of small chunks.
       
  2118 
       
  2119   Each tree contains a power of 2 sized range of chunk sizes (the
       
  2120   smallest is 0x100 <= x < 0x180), which is is divided in half at each
       
  2121   tree level, with the chunks in the smaller half of the range (0x100
       
  2122   <= x < 0x140 for the top nose) in the left subtree and the larger
       
  2123   half (0x140 <= x < 0x180) in the right subtree.  This is, of course,
       
  2124   done by inspecting individual bits.
       
  2125 
       
  2126   Using these rules, each node's left subtree contains all smaller
       
  2127   sizes than its right subtree.  However, the node at the root of each
       
  2128   subtree has no particular ordering relationship to either.  (The
       
  2129   dividing line between the subtree sizes is based on trie relation.)
       
  2130   If we remove the last chunk of a given size from the interior of the
       
  2131   tree, we need to replace it with a leaf node.  The tree ordering
       
  2132   rules permit a node to be replaced by any leaf below it.
       
  2133 
       
  2134   The smallest chunk in a tree (a common operation in a best-fit
       
  2135   allocator) can be found by walking a path to the leftmost leaf in
       
  2136   the tree.  Unlike a usual binary tree, where we follow left child
       
  2137   pointers until we reach a null, here we follow the right child
       
  2138   pointer any time the left one is null, until we reach a leaf with
       
  2139   both child pointers null. The smallest chunk in the tree will be
       
  2140   somewhere along that path.
       
  2141 
       
  2142   The worst case number of steps to add, find, or remove a node is
       
  2143   bounded by the number of bits differentiating chunks within
       
  2144   bins. Under current bin calculations, this ranges from 6 up to 21
       
  2145   (for 32 bit sizes) or up to 53 (for 64 bit sizes). The typical case
       
  2146   is of course much better.
       
  2147 */
       
  2148 
       
  2149 struct malloc_tree_chunk {
       
  2150   /* The first four fields must be compatible with malloc_chunk */
       
  2151   size_t                    prev_foot;
       
  2152   size_t                    head;
       
  2153   struct malloc_tree_chunk* fd;
       
  2154   struct malloc_tree_chunk* bk;
       
  2155 
       
  2156   struct malloc_tree_chunk* child[2];
       
  2157   struct malloc_tree_chunk* parent;
       
  2158   bindex_t                  index;
       
  2159 };
       
  2160 
       
  2161 typedef struct malloc_tree_chunk  tchunk;
       
  2162 typedef struct malloc_tree_chunk* tchunkptr;
       
  2163 typedef struct malloc_tree_chunk* tbinptr; /* The type of bins of trees */
       
  2164 
       
  2165 /* A little helper macro for trees */
       
  2166 #define leftmost_child(t) ((t)->child[0] != 0? (t)->child[0] : (t)->child[1])
       
  2167 
       
  2168 /* ----------------------------- Segments -------------------------------- */
       
  2169 
       
  2170 /*
       
  2171   Each malloc space may include non-contiguous segments, held in a
       
  2172   list headed by an embedded malloc_segment record representing the
       
  2173   top-most space. Segments also include flags holding properties of
       
  2174   the space. Large chunks that are directly allocated by mmap are not
       
  2175   included in this list. They are instead independently created and
       
  2176   destroyed without otherwise keeping track of them.
       
  2177 
       
  2178   Segment management mainly comes into play for spaces allocated by
       
  2179   MMAP.  Any call to MMAP might or might not return memory that is
       
  2180   adjacent to an existing segment.  MORECORE normally contiguously
       
  2181   extends the current space, so this space is almost always adjacent,
       
  2182   which is simpler and faster to deal with. (This is why MORECORE is
       
  2183   used preferentially to MMAP when both are available -- see
       
  2184   sys_alloc.)  When allocating using MMAP, we don't use any of the
       
  2185   hinting mechanisms (inconsistently) supported in various
       
  2186   implementations of unix mmap, or distinguish reserving from
       
  2187   committing memory. Instead, we just ask for space, and exploit
       
  2188   contiguity when we get it.  It is probably possible to do
       
  2189   better than this on some systems, but no general scheme seems
       
  2190   to be significantly better.
       
  2191 
       
  2192   Management entails a simpler variant of the consolidation scheme
       
  2193   used for chunks to reduce fragmentation -- new adjacent memory is
       
  2194   normally prepended or appended to an existing segment. However,
       
  2195   there are limitations compared to chunk consolidation that mostly
       
  2196   reflect the fact that segment processing is relatively infrequent
       
  2197   (occurring only when getting memory from system) and that we
       
  2198   don't expect to have huge numbers of segments:
       
  2199 
       
  2200   * Segments are not indexed, so traversal requires linear scans.  (It
       
  2201     would be possible to index these, but is not worth the extra
       
  2202     overhead and complexity for most programs on most platforms.)
       
  2203   * New segments are only appended to old ones when holding top-most
       
  2204     memory; if they cannot be prepended to others, they are held in
       
  2205     different segments.
       
  2206 
       
  2207   Except for the top-most segment of an mstate, each segment record
       
  2208   is kept at the tail of its segment. Segments are added by pushing
       
  2209   segment records onto the list headed by &mstate.seg for the
       
  2210   containing mstate.
       
  2211 
       
  2212   Segment flags control allocation/merge/deallocation policies:
       
  2213   * If EXTERN_BIT set, then we did not allocate this segment,
       
  2214     and so should not try to deallocate or merge with others.
       
  2215     (This currently holds only for the initial segment passed
       
  2216     into create_mspace_with_base.)
       
  2217   * If IS_MMAPPED_BIT set, the segment may be merged with
       
  2218     other surrounding mmapped segments and trimmed/de-allocated
       
  2219     using munmap.
       
  2220   * If neither bit is set, then the segment was obtained using
       
  2221     MORECORE so can be merged with surrounding MORECORE'd segments
       
  2222     and deallocated/trimmed using MORECORE with negative arguments.
       
  2223 */
       
  2224 
       
  2225 struct malloc_segment {
       
  2226   char*        base;             /* base address */
       
  2227   size_t       size;             /* allocated size */
       
  2228   struct malloc_segment* next;   /* ptr to next segment */
       
  2229   flag_t       sflags;           /* mmap and extern flag */
       
  2230 };
       
  2231 
       
  2232 #define is_mmapped_segment(S)  ((S)->sflags & IS_MMAPPED_BIT)
       
  2233 #define is_extern_segment(S)   ((S)->sflags & EXTERN_BIT)
       
  2234 
       
  2235 typedef struct malloc_segment  msegment;
       
  2236 typedef struct malloc_segment* msegmentptr;
       
  2237 
       
  2238 /* ---------------------------- malloc_state ----------------------------- */
       
  2239 
       
  2240 /*
       
  2241    A malloc_state holds all of the bookkeeping for a space.
       
  2242    The main fields are:
       
  2243 
       
  2244   Top
       
  2245     The topmost chunk of the currently active segment. Its size is
       
  2246     cached in topsize.  The actual size of topmost space is
       
  2247     topsize+TOP_FOOT_SIZE, which includes space reserved for adding
       
  2248     fenceposts and segment records if necessary when getting more
       
  2249     space from the system.  The size at which to autotrim top is
       
  2250     cached from mparams in trim_check, except that it is disabled if
       
  2251     an autotrim fails.
       
  2252 
       
  2253   Designated victim (dv)
       
  2254     This is the preferred chunk for servicing small requests that
       
  2255     don't have exact fits.  It is normally the chunk split off most
       
  2256     recently to service another small request.  Its size is cached in
       
  2257     dvsize. The link fields of this chunk are not maintained since it
       
  2258     is not kept in a bin.
       
  2259 
       
  2260   SmallBins
       
  2261     An array of bin headers for free chunks.  These bins hold chunks
       
  2262     with sizes less than MIN_LARGE_SIZE bytes. Each bin contains
       
  2263     chunks of all the same size, spaced 8 bytes apart.  To simplify
       
  2264     use in double-linked lists, each bin header acts as a malloc_chunk
       
  2265     pointing to the real first node, if it exists (else pointing to
       
  2266     itself).  This avoids special-casing for headers.  But to avoid
       
  2267     waste, we allocate only the fd/bk pointers of bins, and then use
       
  2268     repositioning tricks to treat these as the fields of a chunk.
       
  2269 
       
  2270   TreeBins
       
  2271     Treebins are pointers to the roots of trees holding a range of
       
  2272     sizes. There are 2 equally spaced treebins for each power of two
       
  2273     from TREE_SHIFT to TREE_SHIFT+16. The last bin holds anything
       
  2274     larger.
       
  2275 
       
  2276   Bin maps
       
  2277     There is one bit map for small bins ("smallmap") and one for
       
  2278     treebins ("treemap).  Each bin sets its bit when non-empty, and
       
  2279     clears the bit when empty.  Bit operations are then used to avoid
       
  2280     bin-by-bin searching -- nearly all "search" is done without ever
       
  2281     looking at bins that won't be selected.  The bit maps
       
  2282     conservatively use 32 bits per map word, even if on 64bit system.
       
  2283     For a good description of some of the bit-based techniques used
       
  2284     here, see Henry S. Warren Jr's book "Hacker's Delight" (and
       
  2285     supplement at http://hackersdelight.org/). Many of these are
       
  2286     intended to reduce the branchiness of paths through malloc etc, as
       
  2287     well as to reduce the number of memory locations read or written.
       
  2288 
       
  2289   Segments
       
  2290     A list of segments headed by an embedded malloc_segment record
       
  2291     representing the initial space.
       
  2292 
       
  2293   Address check support
       
  2294     The least_addr field is the least address ever obtained from
       
  2295     MORECORE or MMAP. Attempted frees and reallocs of any address less
       
  2296     than this are trapped (unless INSECURE is defined).
       
  2297 
       
  2298   Magic tag
       
  2299     A cross-check field that should always hold same value as mparams.magic.
       
  2300 
       
  2301   Flags
       
  2302     Bits recording whether to use MMAP, locks, or contiguous MORECORE
       
  2303 
       
  2304   Statistics
       
  2305     Each space keeps track of current and maximum system memory
       
  2306     obtained via MORECORE or MMAP.
       
  2307 
       
  2308   Trim support
       
  2309     Fields holding the amount of unused topmost memory that should trigger
       
  2310     timming, and a counter to force periodic scanning to release unused
       
  2311     non-topmost segments.
       
  2312 
       
  2313   Locking
       
  2314     If USE_LOCKS is defined, the "mutex" lock is acquired and released
       
  2315     around every public call using this mspace.
       
  2316 
       
  2317   Extension support
       
  2318     A void* pointer and a size_t field that can be used to help implement
       
  2319     extensions to this malloc.
       
  2320 */
       
  2321 
       
  2322 /* Bin types, widths and sizes */
       
  2323 #define NSMALLBINS        (32U)
       
  2324 #define NTREEBINS         (32U)
       
  2325 #define SMALLBIN_SHIFT    (3U)
       
  2326 #define SMALLBIN_WIDTH    (SIZE_T_ONE << SMALLBIN_SHIFT)
       
  2327 #define TREEBIN_SHIFT     (8U)
       
  2328 #define MIN_LARGE_SIZE    (SIZE_T_ONE << TREEBIN_SHIFT)
       
  2329 #define MAX_SMALL_SIZE    (MIN_LARGE_SIZE - SIZE_T_ONE)
       
  2330 #define MAX_SMALL_REQUEST (MAX_SMALL_SIZE - CHUNK_ALIGN_MASK - CHUNK_OVERHEAD)
       
  2331 
       
  2332 struct malloc_state {
       
  2333   binmap_t   smallmap;
       
  2334   binmap_t   treemap;
       
  2335   size_t     dvsize;
       
  2336   size_t     topsize;
       
  2337   char*      least_addr;
       
  2338   mchunkptr  dv;
       
  2339   mchunkptr  top;
       
  2340   size_t     trim_check;
       
  2341   size_t     release_checks;
       
  2342   size_t     magic;
       
  2343   mchunkptr  smallbins[(NSMALLBINS+1)*2];
       
  2344   tbinptr    treebins[NTREEBINS];
       
  2345   size_t     footprint;
       
  2346   size_t     max_footprint;
       
  2347   flag_t     mflags;
       
  2348 #if USE_LOCKS
       
  2349   MLOCK_T    mutex;     /* locate lock among fields that rarely change */
       
  2350 #endif /* USE_LOCKS */
       
  2351   msegment   seg;
       
  2352   void*      extp;      /* Unused but available for extensions */
       
  2353   size_t     exts;
       
  2354 };
       
  2355 
       
  2356 typedef struct malloc_state*    mstate;
       
  2357 
       
  2358 /* ------------- Global malloc_state and malloc_params ------------------- */
       
  2359 
       
  2360 /*
       
  2361   malloc_params holds global properties, including those that can be
       
  2362   dynamically set using mallopt. There is a single instance, mparams,
       
  2363   initialized in init_mparams.
       
  2364 */
       
  2365 
       
  2366 struct malloc_params {
       
  2367   size_t magic;
       
  2368   size_t page_size;
       
  2369   size_t granularity;
       
  2370   size_t mmap_threshold;
       
  2371   size_t trim_threshold;
       
  2372   flag_t default_mflags;
       
  2373 };
       
  2374 
       
  2375 static struct malloc_params mparams;
       
  2376 
       
  2377 #if !ONLY_MSPACES
       
  2378 
       
  2379 /* The global malloc_state used for all non-"mspace" calls */
       
  2380 static struct malloc_state _gm_;
       
  2381 #define gm                 (&_gm_)
       
  2382 #define is_global(M)       ((M) == &_gm_)
       
  2383 
       
  2384 #endif /* !ONLY_MSPACES */
       
  2385 
       
  2386 #define is_initialized(M)  ((M)->top != 0)
       
  2387 
       
  2388 /* -------------------------- system alloc setup ------------------------- */
       
  2389 
       
  2390 /* Operations on mflags */
       
  2391 
       
  2392 #define use_lock(M)           ((M)->mflags &   USE_LOCK_BIT)
       
  2393 #define enable_lock(M)        ((M)->mflags |=  USE_LOCK_BIT)
       
  2394 #define disable_lock(M)       ((M)->mflags &= ~USE_LOCK_BIT)
       
  2395 
       
  2396 #define use_mmap(M)           ((M)->mflags &   USE_MMAP_BIT)
       
  2397 #define enable_mmap(M)        ((M)->mflags |=  USE_MMAP_BIT)
       
  2398 #define disable_mmap(M)       ((M)->mflags &= ~USE_MMAP_BIT)
       
  2399 
       
  2400 #define use_noncontiguous(M)  ((M)->mflags &   USE_NONCONTIGUOUS_BIT)
       
  2401 #define disable_contiguous(M) ((M)->mflags |=  USE_NONCONTIGUOUS_BIT)
       
  2402 
       
  2403 #define set_lock(M,L)\
       
  2404  ((M)->mflags = (L)?\
       
  2405   ((M)->mflags | USE_LOCK_BIT) :\
       
  2406   ((M)->mflags & ~USE_LOCK_BIT))
       
  2407 
       
  2408 /* page-align a size */
       
  2409 #define page_align(S)\
       
  2410  (((S) + (mparams.page_size - SIZE_T_ONE)) & ~(mparams.page_size - SIZE_T_ONE))
       
  2411 
       
  2412 /* granularity-align a size */
       
  2413 #define granularity_align(S)\
       
  2414   (((S) + (mparams.granularity - SIZE_T_ONE))\
       
  2415    & ~(mparams.granularity - SIZE_T_ONE))
       
  2416 
       
  2417 
       
  2418 /* For mmap, use granularity alignment on windows, else page-align */
       
  2419 #ifdef WIN32
       
  2420 #define mmap_align(S) granularity_align(S)
       
  2421 #else
       
  2422 #define mmap_align(S) page_align(S)
       
  2423 #endif
       
  2424 
       
  2425 #define is_page_aligned(S)\
       
  2426    (((size_t)(S) & (mparams.page_size - SIZE_T_ONE)) == 0)
       
  2427 #define is_granularity_aligned(S)\
       
  2428    (((size_t)(S) & (mparams.granularity - SIZE_T_ONE)) == 0)
       
  2429 
       
  2430 /*  True if segment S holds address A */
       
  2431 #define segment_holds(S, A)\
       
  2432   ((char*)(A) >= S->base && (char*)(A) < S->base + S->size)
       
  2433 
       
  2434 /* Return segment holding given address */
       
  2435 static msegmentptr segment_holding(mstate m, char* addr) {
       
  2436   msegmentptr sp = &m->seg;
       
  2437   for (;;) {
       
  2438     if (addr >= sp->base && addr < sp->base + sp->size)
       
  2439       return sp;
       
  2440     if ((sp = sp->next) == 0)
       
  2441       return 0;
       
  2442   }
       
  2443 }
       
  2444 
       
  2445 /* Return true if segment contains a segment link */
       
  2446 static int has_segment_link(mstate m, msegmentptr ss) {
       
  2447   msegmentptr sp = &m->seg;
       
  2448   for (;;) {
       
  2449     if ((char*)sp >= ss->base && (char*)sp < ss->base + ss->size)
       
  2450       return 1;
       
  2451     if ((sp = sp->next) == 0)
       
  2452       return 0;
       
  2453   }
       
  2454 }
       
  2455 
       
  2456 #ifndef MORECORE_CANNOT_TRIM
       
  2457 #define should_trim(M,s)  ((s) > (M)->trim_check)
       
  2458 #else  /* MORECORE_CANNOT_TRIM */
       
  2459 #define should_trim(M,s)  (0)
       
  2460 #endif /* MORECORE_CANNOT_TRIM */
       
  2461 
       
  2462 /*
       
  2463   TOP_FOOT_SIZE is padding at the end of a segment, including space
       
  2464   that may be needed to place segment records and fenceposts when new
       
  2465   noncontiguous segments are added.
       
  2466 */
       
  2467 #define TOP_FOOT_SIZE\
       
  2468   (align_offset(chunk2mem(0))+pad_request(sizeof(struct malloc_segment))+MIN_CHUNK_SIZE)
       
  2469 
       
  2470 
       
  2471 /* -------------------------------  Hooks -------------------------------- */
       
  2472 
       
  2473 /*
       
  2474   PREACTION should be defined to return 0 on success, and nonzero on
       
  2475   failure. If you are not using locking, you can redefine these to do
       
  2476   anything you like.
       
  2477 */
       
  2478 
       
  2479 #if USE_LOCKS
       
  2480 
       
  2481 /* Ensure locks are initialized */
       
  2482 #define GLOBALLY_INITIALIZE() (mparams.page_size == 0 && init_mparams())
       
  2483 
       
  2484 #define PREACTION(M)  ((GLOBALLY_INITIALIZE() || use_lock(M))? ACQUIRE_LOCK(&(M)->mutex) : 0)
       
  2485 #define POSTACTION(M) { if (use_lock(M)) RELEASE_LOCK(&(M)->mutex); }
       
  2486 #else /* USE_LOCKS */
       
  2487 
       
  2488 #ifndef PREACTION
       
  2489 #define PREACTION(M) (0)
       
  2490 #endif  /* PREACTION */
       
  2491 
       
  2492 #ifndef POSTACTION
       
  2493 #define POSTACTION(M)
       
  2494 #endif  /* POSTACTION */
       
  2495 
       
  2496 #endif /* USE_LOCKS */
       
  2497 
       
  2498 /*
       
  2499   CORRUPTION_ERROR_ACTION is triggered upon detected bad addresses.
       
  2500   USAGE_ERROR_ACTION is triggered on detected bad frees and
       
  2501   reallocs. The argument p is an address that might have triggered the
       
  2502   fault. It is ignored by the two predefined actions, but might be
       
  2503   useful in custom actions that try to help diagnose errors.
       
  2504 */
       
  2505 
       
  2506 #if PROCEED_ON_ERROR
       
  2507 
       
  2508 /* A count of the number of corruption errors causing resets */
       
  2509 int malloc_corruption_error_count;
       
  2510 
       
  2511 /* default corruption action */
       
  2512 static void reset_on_error(mstate m);
       
  2513 
       
  2514 #define CORRUPTION_ERROR_ACTION(m)  reset_on_error(m)
       
  2515 #define USAGE_ERROR_ACTION(m, p)
       
  2516 
       
  2517 #else /* PROCEED_ON_ERROR */
       
  2518 
       
  2519 #ifndef CORRUPTION_ERROR_ACTION
       
  2520 #define CORRUPTION_ERROR_ACTION(m) ABORT
       
  2521 #endif /* CORRUPTION_ERROR_ACTION */
       
  2522 
       
  2523 #ifndef USAGE_ERROR_ACTION
       
  2524 #define USAGE_ERROR_ACTION(m,p) ABORT
       
  2525 #endif /* USAGE_ERROR_ACTION */
       
  2526 
       
  2527 #endif /* PROCEED_ON_ERROR */
       
  2528 
       
  2529 /* -------------------------- Debugging setup ---------------------------- */
       
  2530 
       
  2531 #if ! DEBUG
       
  2532 
       
  2533 #define check_free_chunk(M,P)
       
  2534 #define check_inuse_chunk(M,P)
       
  2535 #define check_malloced_chunk(M,P,N)
       
  2536 #define check_mmapped_chunk(M,P)
       
  2537 #define check_malloc_state(M)
       
  2538 #define check_top_chunk(M,P)
       
  2539 
       
  2540 #else /* DEBUG */
       
  2541 #define check_free_chunk(M,P)       do_check_free_chunk(M,P)
       
  2542 #define check_inuse_chunk(M,P)      do_check_inuse_chunk(M,P)
       
  2543 #define check_top_chunk(M,P)        do_check_top_chunk(M,P)
       
  2544 #define check_malloced_chunk(M,P,N) do_check_malloced_chunk(M,P,N)
       
  2545 #define check_mmapped_chunk(M,P)    do_check_mmapped_chunk(M,P)
       
  2546 #define check_malloc_state(M)       do_check_malloc_state(M)
       
  2547 
       
  2548 static void   do_check_any_chunk(mstate m, mchunkptr p);
       
  2549 static void   do_check_top_chunk(mstate m, mchunkptr p);
       
  2550 static void   do_check_mmapped_chunk(mstate m, mchunkptr p);
       
  2551 static void   do_check_inuse_chunk(mstate m, mchunkptr p);
       
  2552 static void   do_check_free_chunk(mstate m, mchunkptr p);
       
  2553 static void   do_check_malloced_chunk(mstate m, void* mem, size_t s);
       
  2554 static void   do_check_tree(mstate m, tchunkptr t);
       
  2555 static void   do_check_treebin(mstate m, bindex_t i);
       
  2556 static void   do_check_smallbin(mstate m, bindex_t i);
       
  2557 static void   do_check_malloc_state(mstate m);
       
  2558 static int    bin_find(mstate m, mchunkptr x);
       
  2559 static size_t traverse_and_check(mstate m);
       
  2560 #endif /* DEBUG */
       
  2561 
       
  2562 /* ---------------------------- Indexing Bins ---------------------------- */
       
  2563 
       
  2564 #define is_small(s)         (((s) >> SMALLBIN_SHIFT) < NSMALLBINS)
       
  2565 #define small_index(s)      ((s)  >> SMALLBIN_SHIFT)
       
  2566 #define small_index2size(i) ((i)  << SMALLBIN_SHIFT)
       
  2567 #define MIN_SMALL_INDEX     (small_index(MIN_CHUNK_SIZE))
       
  2568 
       
  2569 /* addressing by index. See above about smallbin repositioning */
       
  2570 #define smallbin_at(M, i)   ((sbinptr)((char*)&((M)->smallbins[(i)<<1])))
       
  2571 #define treebin_at(M,i)     (&((M)->treebins[i]))
       
  2572 
       
  2573 /* assign tree index for size S to variable I */
       
  2574 #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))
       
  2575 #define compute_tree_index(S, I)\
       
  2576 {\
       
  2577   unsigned int X = S >> TREEBIN_SHIFT;\
       
  2578   if (X == 0)\
       
  2579     I = 0;\
       
  2580   else if (X > 0xFFFF)\
       
  2581     I = NTREEBINS-1;\
       
  2582   else {\
       
  2583     unsigned int K;\
       
  2584     __asm__("bsrl\t%1, %0\n\t" : "=r" (K) : "g"  (X));\
       
  2585     I =  (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\
       
  2586   }\
       
  2587 }
       
  2588 
       
  2589 #elif defined(_MSC_VER) && _MSC_VER>=1300
       
  2590 #define compute_tree_index(S, I)\
       
  2591 {\
       
  2592   size_t X = S >> TREEBIN_SHIFT;\
       
  2593   if (X == 0)\
       
  2594     I = 0;\
       
  2595   else if (X > 0xFFFF)\
       
  2596     I = NTREEBINS-1;\
       
  2597   else {\
       
  2598     unsigned int K;\
       
  2599     _BitScanReverse((DWORD *) &K, X);\
       
  2600     I =  (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\
       
  2601   }\
       
  2602 }
       
  2603 #else /* GNUC */
       
  2604 #define compute_tree_index(S, I)\
       
  2605 {\
       
  2606   size_t X = S >> TREEBIN_SHIFT;\
       
  2607   if (X == 0)\
       
  2608     I = 0;\
       
  2609   else if (X > 0xFFFF)\
       
  2610     I = NTREEBINS-1;\
       
  2611   else {\
       
  2612     unsigned int Y = (unsigned int)X;\
       
  2613     unsigned int N = ((Y - 0x100) >> 16) & 8;\
       
  2614     unsigned int K = (((Y <<= N) - 0x1000) >> 16) & 4;\
       
  2615     N += K;\
       
  2616     N += K = (((Y <<= K) - 0x4000) >> 16) & 2;\
       
  2617     K = 14 - N + ((Y <<= K) >> 15);\
       
  2618     I = (K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1));\
       
  2619   }\
       
  2620 }
       
  2621 #endif /* GNUC */
       
  2622 
       
  2623 /* Bit representing maximum resolved size in a treebin at i */
       
  2624 #define bit_for_tree_index(i) \
       
  2625    (i == NTREEBINS-1)? (SIZE_T_BITSIZE-1) : (((i) >> 1) + TREEBIN_SHIFT - 2)
       
  2626 
       
  2627 /* Shift placing maximum resolved bit in a treebin at i as sign bit */
       
  2628 #define leftshift_for_tree_index(i) \
       
  2629    ((i == NTREEBINS-1)? 0 : \
       
  2630     ((SIZE_T_BITSIZE-SIZE_T_ONE) - (((i) >> 1) + TREEBIN_SHIFT - 2)))
       
  2631 
       
  2632 /* The size of the smallest chunk held in bin with index i */
       
  2633 #define minsize_for_tree_index(i) \
       
  2634    ((SIZE_T_ONE << (((i) >> 1) + TREEBIN_SHIFT)) |  \
       
  2635    (((size_t)((i) & SIZE_T_ONE)) << (((i) >> 1) + TREEBIN_SHIFT - 1)))
       
  2636 
       
  2637 
       
  2638 /* ------------------------ Operations on bin maps ----------------------- */
       
  2639 
       
  2640 /* bit corresponding to given index */
       
  2641 #define idx2bit(i)              ((binmap_t)(1) << (i))
       
  2642 
       
  2643 /* Mark/Clear bits with given index */
       
  2644 #define mark_smallmap(M,i)      ((M)->smallmap |=  idx2bit(i))
       
  2645 #define clear_smallmap(M,i)     ((M)->smallmap &= ~idx2bit(i))
       
  2646 #define smallmap_is_marked(M,i) ((M)->smallmap &   idx2bit(i))
       
  2647 
       
  2648 #define mark_treemap(M,i)       ((M)->treemap  |=  idx2bit(i))
       
  2649 #define clear_treemap(M,i)      ((M)->treemap  &= ~idx2bit(i))
       
  2650 #define treemap_is_marked(M,i)  ((M)->treemap  &   idx2bit(i))
       
  2651 
       
  2652 /* index corresponding to given bit */
       
  2653 
       
  2654 #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))
       
  2655 #define compute_bit2idx(X, I)\
       
  2656 {\
       
  2657   unsigned int J;\
       
  2658   __asm__("bsfl\t%1, %0\n\t" : "=r" (J) : "g" (X));\
       
  2659   I = (bindex_t)J;\
       
  2660 }
       
  2661 #elif defined(_MSC_VER) && _MSC_VER>=1300
       
  2662 #define compute_bit2idx(X, I)\
       
  2663 {\
       
  2664   unsigned int J;\
       
  2665   _BitScanForward((DWORD *) &J, X);\
       
  2666   I = (bindex_t)J;\
       
  2667 }
       
  2668 
       
  2669 #else /* GNUC */
       
  2670 #if  USE_BUILTIN_FFS
       
  2671 #define compute_bit2idx(X, I) I = ffs(X)-1
       
  2672 
       
  2673 #else /* USE_BUILTIN_FFS */
       
  2674 #define compute_bit2idx(X, I)\
       
  2675 {\
       
  2676   unsigned int Y = X - 1;\
       
  2677   unsigned int K = Y >> (16-4) & 16;\
       
  2678   unsigned int N = K;        Y >>= K;\
       
  2679   N += K = Y >> (8-3) &  8;  Y >>= K;\
       
  2680   N += K = Y >> (4-2) &  4;  Y >>= K;\
       
  2681   N += K = Y >> (2-1) &  2;  Y >>= K;\
       
  2682   N += K = Y >> (1-0) &  1;  Y >>= K;\
       
  2683   I = (bindex_t)(N + Y);\
       
  2684 }
       
  2685 #endif /* USE_BUILTIN_FFS */
       
  2686 #endif /* GNUC */
       
  2687 
       
  2688 /* isolate the least set bit of a bitmap */
       
  2689 #define least_bit(x)         ((x) & -(x))
       
  2690 
       
  2691 /* mask with all bits to left of least bit of x on */
       
  2692 #define left_bits(x)         ((x<<1) | -(x<<1))
       
  2693 
       
  2694 /* mask with all bits to left of or equal to least bit of x on */
       
  2695 #define same_or_left_bits(x) ((x) | -(x))
       
  2696 
       
  2697 
       
  2698 /* ----------------------- Runtime Check Support ------------------------- */
       
  2699 
       
  2700 /*
       
  2701   For security, the main invariant is that malloc/free/etc never
       
  2702   writes to a static address other than malloc_state, unless static
       
  2703   malloc_state itself has been corrupted, which cannot occur via
       
  2704   malloc (because of these checks). In essence this means that we
       
  2705   believe all pointers, sizes, maps etc held in malloc_state, but
       
  2706   check all of those linked or offsetted from other embedded data
       
  2707   structures.  These checks are interspersed with main code in a way
       
  2708   that tends to minimize their run-time cost.
       
  2709 
       
  2710   When FOOTERS is defined, in addition to range checking, we also
       
  2711   verify footer fields of inuse chunks, which can be used guarantee
       
  2712   that the mstate controlling malloc/free is intact.  This is a
       
  2713   streamlined version of the approach described by William Robertson
       
  2714   et al in "Run-time Detection of Heap-based Overflows" LISA'03
       
  2715   http://www.usenix.org/events/lisa03/tech/robertson.html The footer
       
  2716   of an inuse chunk holds the xor of its mstate and a random seed,
       
  2717   that is checked upon calls to free() and realloc().  This is
       
  2718   (probablistically) unguessable from outside the program, but can be
       
  2719   computed by any code successfully malloc'ing any chunk, so does not
       
  2720   itself provide protection against code that has already broken
       
  2721   security through some other means.  Unlike Robertson et al, we
       
  2722   always dynamically check addresses of all offset chunks (previous,
       
  2723   next, etc). This turns out to be cheaper than relying on hashes.
       
  2724 */
       
  2725 
       
  2726 #if !INSECURE
       
  2727 /* Check if address a is at least as high as any from MORECORE or MMAP */
       
  2728 #define ok_address(M, a) ((char*)(a) >= (M)->least_addr)
       
  2729 /* Check if address of next chunk n is higher than base chunk p */
       
  2730 #define ok_next(p, n)    ((char*)(p) < (char*)(n))
       
  2731 /* Check if p has its cinuse bit on */
       
  2732 #define ok_cinuse(p)     cinuse(p)
       
  2733 /* Check if p has its pinuse bit on */
       
  2734 #define ok_pinuse(p)     pinuse(p)
       
  2735 
       
  2736 #else /* !INSECURE */
       
  2737 #define ok_address(M, a) (1)
       
  2738 #define ok_next(b, n)    (1)
       
  2739 #define ok_cinuse(p)     (1)
       
  2740 #define ok_pinuse(p)     (1)
       
  2741 #endif /* !INSECURE */
       
  2742 
       
  2743 #if (FOOTERS && !INSECURE)
       
  2744 /* Check if (alleged) mstate m has expected magic field */
       
  2745 #define ok_magic(M)      ((M)->magic == mparams.magic)
       
  2746 #else  /* (FOOTERS && !INSECURE) */
       
  2747 #define ok_magic(M)      (1)
       
  2748 #endif /* (FOOTERS && !INSECURE) */
       
  2749 
       
  2750 
       
  2751 /* In gcc, use __builtin_expect to minimize impact of checks */
       
  2752 #if !INSECURE
       
  2753 #if defined(__GNUC__) && __GNUC__ >= 3
       
  2754 #define RTCHECK(e)  __builtin_expect(e, 1)
       
  2755 #else /* GNUC */
       
  2756 #define RTCHECK(e)  (e)
       
  2757 #endif /* GNUC */
       
  2758 #else /* !INSECURE */
       
  2759 #define RTCHECK(e)  (1)
       
  2760 #endif /* !INSECURE */
       
  2761 
       
  2762 /* macros to set up inuse chunks with or without footers */
       
  2763 
       
  2764 #if !FOOTERS
       
  2765 
       
  2766 #define mark_inuse_foot(M,p,s)
       
  2767 
       
  2768 /* Set cinuse bit and pinuse bit of next chunk */
       
  2769 #define set_inuse(M,p,s)\
       
  2770   ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\
       
  2771   ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT)
       
  2772 
       
  2773 /* Set cinuse and pinuse of this chunk and pinuse of next chunk */
       
  2774 #define set_inuse_and_pinuse(M,p,s)\
       
  2775   ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\
       
  2776   ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT)
       
  2777 
       
  2778 /* Set size, cinuse and pinuse bit of this chunk */
       
  2779 #define set_size_and_pinuse_of_inuse_chunk(M, p, s)\
       
  2780   ((p)->head = (s|PINUSE_BIT|CINUSE_BIT))
       
  2781 
       
  2782 #else /* FOOTERS */
       
  2783 
       
  2784 /* Set foot of inuse chunk to be xor of mstate and seed */
       
  2785 #define mark_inuse_foot(M,p,s)\
       
  2786   (((mchunkptr)((char*)(p) + (s)))->prev_foot = ((size_t)(M) ^ mparams.magic))
       
  2787 
       
  2788 #define get_mstate_for(p)\
       
  2789   ((mstate)(((mchunkptr)((char*)(p) +\
       
  2790     (chunksize(p))))->prev_foot ^ mparams.magic))
       
  2791 
       
  2792 #define set_inuse(M,p,s)\
       
  2793   ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\
       
  2794   (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT), \
       
  2795   mark_inuse_foot(M,p,s))
       
  2796 
       
  2797 #define set_inuse_and_pinuse(M,p,s)\
       
  2798   ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\
       
  2799   (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT),\
       
  2800  mark_inuse_foot(M,p,s))
       
  2801 
       
  2802 #define set_size_and_pinuse_of_inuse_chunk(M, p, s)\
       
  2803   ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\
       
  2804   mark_inuse_foot(M, p, s))
       
  2805 
       
  2806 #endif /* !FOOTERS */
       
  2807 
       
  2808 /* ---------------------------- setting mparams -------------------------- */
       
  2809 
       
  2810 /* Initialize mparams */
       
  2811 static int init_mparams(void) {
       
  2812   if (mparams.page_size == 0) {
       
  2813     size_t s;
       
  2814 
       
  2815     mparams.mmap_threshold = DEFAULT_MMAP_THRESHOLD;
       
  2816     mparams.trim_threshold = DEFAULT_TRIM_THRESHOLD;
       
  2817 #if MORECORE_CONTIGUOUS
       
  2818     mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT;
       
  2819 #else  /* MORECORE_CONTIGUOUS */
       
  2820     mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT|USE_NONCONTIGUOUS_BIT;
       
  2821 #endif /* MORECORE_CONTIGUOUS */
       
  2822 
       
  2823 #if (FOOTERS && !INSECURE)
       
  2824     {
       
  2825 #if USE_DEV_RANDOM
       
  2826       int fd;
       
  2827       unsigned char buf[sizeof(size_t)];
       
  2828       /* Try to use /dev/urandom, else fall back on using time */
       
  2829       if ((fd = open("/dev/urandom", O_RDONLY)) >= 0 &&
       
  2830           read(fd, buf, sizeof(buf)) == sizeof(buf)) {
       
  2831         s = *((size_t *) buf);
       
  2832         close(fd);
       
  2833       }
       
  2834       else
       
  2835 #endif /* USE_DEV_RANDOM */
       
  2836         s = (size_t)(time(0) ^ (size_t)0x55555555U);
       
  2837 
       
  2838       s |= (size_t)8U;    /* ensure nonzero */
       
  2839       s &= ~(size_t)7U;   /* improve chances of fault for bad values */
       
  2840 
       
  2841     }
       
  2842 #else /* (FOOTERS && !INSECURE) */
       
  2843     s = (size_t)0x58585858U;
       
  2844 #endif /* (FOOTERS && !INSECURE) */
       
  2845     ACQUIRE_MAGIC_INIT_LOCK();
       
  2846     if (mparams.magic == 0) {
       
  2847       mparams.magic = s;
       
  2848 #if !ONLY_MSPACES
       
  2849       /* Set up lock for main malloc area */
       
  2850       INITIAL_LOCK(&gm->mutex);
       
  2851       gm->mflags = mparams.default_mflags;
       
  2852 #endif
       
  2853     }
       
  2854     RELEASE_MAGIC_INIT_LOCK();
       
  2855 
       
  2856 #ifndef WIN32
       
  2857     mparams.page_size = malloc_getpagesize;
       
  2858     mparams.granularity = ((DEFAULT_GRANULARITY != 0)?
       
  2859                            DEFAULT_GRANULARITY : mparams.page_size);
       
  2860 #else /* WIN32 */
       
  2861     {
       
  2862       SYSTEM_INFO system_info;
       
  2863       GetSystemInfo(&system_info);
       
  2864       mparams.page_size = system_info.dwPageSize;
       
  2865       mparams.granularity = system_info.dwAllocationGranularity;
       
  2866     }
       
  2867 #endif /* WIN32 */
       
  2868 
       
  2869     /* Sanity-check configuration:
       
  2870        size_t must be unsigned and as wide as pointer type.
       
  2871        ints must be at least 4 bytes.
       
  2872        alignment must be at least 8.
       
  2873        Alignment, min chunk size, and page size must all be powers of 2.
       
  2874     */
       
  2875     if ((sizeof(size_t) != sizeof(char*)) ||
       
  2876         (MAX_SIZE_T < MIN_CHUNK_SIZE)  ||
       
  2877         (sizeof(int) < 4)  ||
       
  2878         (MALLOC_ALIGNMENT < (size_t)8U) ||
       
  2879         ((MALLOC_ALIGNMENT    & (MALLOC_ALIGNMENT-SIZE_T_ONE))    != 0) ||
       
  2880         ((MCHUNK_SIZE         & (MCHUNK_SIZE-SIZE_T_ONE))         != 0) ||
       
  2881         ((mparams.granularity & (mparams.granularity-SIZE_T_ONE)) != 0) ||
       
  2882         ((mparams.page_size   & (mparams.page_size-SIZE_T_ONE))   != 0))
       
  2883       ABORT;
       
  2884   }
       
  2885   return 0;
       
  2886 }
       
  2887 
       
  2888 /* support for mallopt */
       
  2889 static int change_mparam(int param_number, int value) {
       
  2890   size_t val = (size_t)value;
       
  2891   init_mparams();
       
  2892   switch(param_number) {
       
  2893   case M_TRIM_THRESHOLD:
       
  2894     mparams.trim_threshold = val;
       
  2895     return 1;
       
  2896   case M_GRANULARITY:
       
  2897     if (val >= mparams.page_size && ((val & (val-1)) == 0)) {
       
  2898       mparams.granularity = val;
       
  2899       return 1;
       
  2900     }
       
  2901     else
       
  2902       return 0;
       
  2903   case M_MMAP_THRESHOLD:
       
  2904     mparams.mmap_threshold = val;
       
  2905     return 1;
       
  2906   default:
       
  2907     return 0;
       
  2908   }
       
  2909 }
       
  2910 
       
  2911 #if DEBUG
       
  2912 /* ------------------------- Debugging Support --------------------------- */
       
  2913 
       
  2914 /* Check properties of any chunk, whether free, inuse, mmapped etc  */
       
  2915 static void do_check_any_chunk(mstate m, mchunkptr p) {
       
  2916   assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD));
       
  2917   assert(ok_address(m, p));
       
  2918 }
       
  2919 
       
  2920 /* Check properties of top chunk */
       
  2921 static void do_check_top_chunk(mstate m, mchunkptr p) {
       
  2922   msegmentptr sp = segment_holding(m, (char*)p);
       
  2923   size_t  sz = p->head & ~INUSE_BITS; /* third-lowest bit can be set! */
       
  2924   assert(sp != 0);
       
  2925   assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD));
       
  2926   assert(ok_address(m, p));
       
  2927   assert(sz == m->topsize);
       
  2928   assert(sz > 0);
       
  2929   assert(sz == ((sp->base + sp->size) - (char*)p) - TOP_FOOT_SIZE);
       
  2930   assert(pinuse(p));
       
  2931   assert(!pinuse(chunk_plus_offset(p, sz)));
       
  2932 }
       
  2933 
       
  2934 /* Check properties of (inuse) mmapped chunks */
       
  2935 static void do_check_mmapped_chunk(mstate m, mchunkptr p) {
       
  2936   size_t  sz = chunksize(p);
       
  2937   size_t len = (sz + (p->prev_foot & ~IS_MMAPPED_BIT) + MMAP_FOOT_PAD);
       
  2938   assert(is_mmapped(p));
       
  2939   assert(use_mmap(m));
       
  2940   assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD));
       
  2941   assert(ok_address(m, p));
       
  2942   assert(!is_small(sz));
       
  2943   assert((len & (mparams.page_size-SIZE_T_ONE)) == 0);
       
  2944   assert(chunk_plus_offset(p, sz)->head == FENCEPOST_HEAD);
       
  2945   assert(chunk_plus_offset(p, sz+SIZE_T_SIZE)->head == 0);
       
  2946 }
       
  2947 
       
  2948 /* Check properties of inuse chunks */
       
  2949 static void do_check_inuse_chunk(mstate m, mchunkptr p) {
       
  2950   do_check_any_chunk(m, p);
       
  2951   assert(cinuse(p));
       
  2952   assert(next_pinuse(p));
       
  2953   /* If not pinuse and not mmapped, previous chunk has OK offset */
       
  2954   assert(is_mmapped(p) || pinuse(p) || next_chunk(prev_chunk(p)) == p);
       
  2955   if (is_mmapped(p))
       
  2956     do_check_mmapped_chunk(m, p);
       
  2957 }
       
  2958 
       
  2959 /* Check properties of free chunks */
       
  2960 static void do_check_free_chunk(mstate m, mchunkptr p) {
       
  2961   size_t sz = chunksize(p);
       
  2962   mchunkptr next = chunk_plus_offset(p, sz);
       
  2963   do_check_any_chunk(m, p);
       
  2964   assert(!cinuse(p));
       
  2965   assert(!next_pinuse(p));
       
  2966   assert (!is_mmapped(p));
       
  2967   if (p != m->dv && p != m->top) {
       
  2968     if (sz >= MIN_CHUNK_SIZE) {
       
  2969       assert((sz & CHUNK_ALIGN_MASK) == 0);
       
  2970       assert(is_aligned(chunk2mem(p)));
       
  2971       assert(next->prev_foot == sz);
       
  2972       assert(pinuse(p));
       
  2973       assert (next == m->top || cinuse(next));
       
  2974       assert(p->fd->bk == p);
       
  2975       assert(p->bk->fd == p);
       
  2976     }
       
  2977     else  /* markers are always of size SIZE_T_SIZE */
       
  2978       assert(sz == SIZE_T_SIZE);
       
  2979   }
       
  2980 }
       
  2981 
       
  2982 /* Check properties of malloced chunks at the point they are malloced */
       
  2983 static void do_check_malloced_chunk(mstate m, void* mem, size_t s) {
       
  2984   if (mem != 0) {
       
  2985     mchunkptr p = mem2chunk(mem);
       
  2986     size_t sz = p->head & ~(PINUSE_BIT|CINUSE_BIT);
       
  2987     do_check_inuse_chunk(m, p);
       
  2988     assert((sz & CHUNK_ALIGN_MASK) == 0);
       
  2989     assert(sz >= MIN_CHUNK_SIZE);
       
  2990     assert(sz >= s);
       
  2991     /* unless mmapped, size is less than MIN_CHUNK_SIZE more than request */
       
  2992     assert(is_mmapped(p) || sz < (s + MIN_CHUNK_SIZE));
       
  2993   }
       
  2994 }
       
  2995 
       
  2996 /* Check a tree and its subtrees.  */
       
  2997 static void do_check_tree(mstate m, tchunkptr t) {
       
  2998   tchunkptr head = 0;
       
  2999   tchunkptr u = t;
       
  3000   bindex_t tindex = t->index;
       
  3001   size_t tsize = chunksize(t);
       
  3002   bindex_t idx;
       
  3003   compute_tree_index(tsize, idx);
       
  3004   assert(tindex == idx);
       
  3005   assert(tsize >= MIN_LARGE_SIZE);
       
  3006   assert(tsize >= minsize_for_tree_index(idx));
       
  3007   assert((idx == NTREEBINS-1) || (tsize < minsize_for_tree_index((idx+1))));
       
  3008 
       
  3009   do { /* traverse through chain of same-sized nodes */
       
  3010     do_check_any_chunk(m, ((mchunkptr)u));
       
  3011     assert(u->index == tindex);
       
  3012     assert(chunksize(u) == tsize);
       
  3013     assert(!cinuse(u));
       
  3014     assert(!next_pinuse(u));
       
  3015     assert(u->fd->bk == u);
       
  3016     assert(u->bk->fd == u);
       
  3017     if (u->parent == 0) {
       
  3018       assert(u->child[0] == 0);
       
  3019       assert(u->child[1] == 0);
       
  3020     }
       
  3021     else {
       
  3022       assert(head == 0); /* only one node on chain has parent */
       
  3023       head = u;
       
  3024       assert(u->parent != u);
       
  3025       assert (u->parent->child[0] == u ||
       
  3026               u->parent->child[1] == u ||
       
  3027               *((tbinptr*)(u->parent)) == u);
       
  3028       if (u->child[0] != 0) {
       
  3029         assert(u->child[0]->parent == u);
       
  3030         assert(u->child[0] != u);
       
  3031         do_check_tree(m, u->child[0]);
       
  3032       }
       
  3033       if (u->child[1] != 0) {
       
  3034         assert(u->child[1]->parent == u);
       
  3035         assert(u->child[1] != u);
       
  3036         do_check_tree(m, u->child[1]);
       
  3037       }
       
  3038       if (u->child[0] != 0 && u->child[1] != 0) {
       
  3039         assert(chunksize(u->child[0]) < chunksize(u->child[1]));
       
  3040       }
       
  3041     }
       
  3042     u = u->fd;
       
  3043   } while (u != t);
       
  3044   assert(head != 0);
       
  3045 }
       
  3046 
       
  3047 /*  Check all the chunks in a treebin.  */
       
  3048 static void do_check_treebin(mstate m, bindex_t i) {
       
  3049   tbinptr* tb = treebin_at(m, i);
       
  3050   tchunkptr t = *tb;
       
  3051   int empty = (m->treemap & (1U << i)) == 0;
       
  3052   if (t == 0)
       
  3053     assert(empty);
       
  3054   if (!empty)
       
  3055     do_check_tree(m, t);
       
  3056 }
       
  3057 
       
  3058 /*  Check all the chunks in a smallbin.  */
       
  3059 static void do_check_smallbin(mstate m, bindex_t i) {
       
  3060   sbinptr b = smallbin_at(m, i);
       
  3061   mchunkptr p = b->bk;
       
  3062   unsigned int empty = (m->smallmap & (1U << i)) == 0;
       
  3063   if (p == b)
       
  3064     assert(empty);
       
  3065   if (!empty) {
       
  3066     for (; p != b; p = p->bk) {
       
  3067       size_t size = chunksize(p);
       
  3068       mchunkptr q;
       
  3069       /* each chunk claims to be free */
       
  3070       do_check_free_chunk(m, p);
       
  3071       /* chunk belongs in bin */
       
  3072       assert(small_index(size) == i);
       
  3073       assert(p->bk == b || chunksize(p->bk) == chunksize(p));
       
  3074       /* chunk is followed by an inuse chunk */
       
  3075       q = next_chunk(p);
       
  3076       if (q->head != FENCEPOST_HEAD)
       
  3077         do_check_inuse_chunk(m, q);
       
  3078     }
       
  3079   }
       
  3080 }
       
  3081 
       
  3082 /* Find x in a bin. Used in other check functions. */
       
  3083 static int bin_find(mstate m, mchunkptr x) {
       
  3084   size_t size = chunksize(x);
       
  3085   if (is_small(size)) {
       
  3086     bindex_t sidx = small_index(size);
       
  3087     sbinptr b = smallbin_at(m, sidx);
       
  3088     if (smallmap_is_marked(m, sidx)) {
       
  3089       mchunkptr p = b;
       
  3090       do {
       
  3091         if (p == x)
       
  3092           return 1;
       
  3093       } while ((p = p->fd) != b);
       
  3094     }
       
  3095   }
       
  3096   else {
       
  3097     bindex_t tidx;
       
  3098     compute_tree_index(size, tidx);
       
  3099     if (treemap_is_marked(m, tidx)) {
       
  3100       tchunkptr t = *treebin_at(m, tidx);
       
  3101       size_t sizebits = size << leftshift_for_tree_index(tidx);
       
  3102       while (t != 0 && chunksize(t) != size) {
       
  3103         t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1];
       
  3104         sizebits <<= 1;
       
  3105       }
       
  3106       if (t != 0) {
       
  3107         tchunkptr u = t;
       
  3108         do {
       
  3109           if (u == (tchunkptr)x)
       
  3110             return 1;
       
  3111         } while ((u = u->fd) != t);
       
  3112       }
       
  3113     }
       
  3114   }
       
  3115   return 0;
       
  3116 }
       
  3117 
       
  3118 /* Traverse each chunk and check it; return total */
       
  3119 static size_t traverse_and_check(mstate m) {
       
  3120   size_t sum = 0;
       
  3121   if (is_initialized(m)) {
       
  3122     msegmentptr s = &m->seg;
       
  3123     sum += m->topsize + TOP_FOOT_SIZE;
       
  3124     while (s != 0) {
       
  3125       mchunkptr q = align_as_chunk(s->base);
       
  3126       mchunkptr lastq = 0;
       
  3127       assert(pinuse(q));
       
  3128       while (segment_holds(s, q) &&
       
  3129              q != m->top && q->head != FENCEPOST_HEAD) {
       
  3130         sum += chunksize(q);
       
  3131         if (cinuse(q)) {
       
  3132           assert(!bin_find(m, q));
       
  3133           do_check_inuse_chunk(m, q);
       
  3134         }
       
  3135         else {
       
  3136           assert(q == m->dv || bin_find(m, q));
       
  3137           assert(lastq == 0 || cinuse(lastq)); /* Not 2 consecutive free */
       
  3138           do_check_free_chunk(m, q);
       
  3139         }
       
  3140         lastq = q;
       
  3141         q = next_chunk(q);
       
  3142       }
       
  3143       s = s->next;
       
  3144     }
       
  3145   }
       
  3146   return sum;
       
  3147 }
       
  3148 
       
  3149 /* Check all properties of malloc_state. */
       
  3150 static void do_check_malloc_state(mstate m) {
       
  3151   bindex_t i;
       
  3152   size_t total;
       
  3153   /* check bins */
       
  3154   for (i = 0; i < NSMALLBINS; ++i)
       
  3155     do_check_smallbin(m, i);
       
  3156   for (i = 0; i < NTREEBINS; ++i)
       
  3157     do_check_treebin(m, i);
       
  3158 
       
  3159   if (m->dvsize != 0) { /* check dv chunk */
       
  3160     do_check_any_chunk(m, m->dv);
       
  3161     assert(m->dvsize == chunksize(m->dv));
       
  3162     assert(m->dvsize >= MIN_CHUNK_SIZE);
       
  3163     assert(bin_find(m, m->dv) == 0);
       
  3164   }
       
  3165 
       
  3166   if (m->top != 0) {   /* check top chunk */
       
  3167     do_check_top_chunk(m, m->top);
       
  3168     /*assert(m->topsize == chunksize(m->top)); redundant */
       
  3169     assert(m->topsize > 0);
       
  3170     assert(bin_find(m, m->top) == 0);
       
  3171   }
       
  3172 
       
  3173   total = traverse_and_check(m);
       
  3174   assert(total <= m->footprint);
       
  3175   assert(m->footprint <= m->max_footprint);
       
  3176 }
       
  3177 #endif /* DEBUG */
       
  3178 
       
  3179 /* ----------------------------- statistics ------------------------------ */
       
  3180 
       
  3181 #if !NO_MALLINFO
       
  3182 static struct mallinfo internal_mallinfo(mstate m) {
       
  3183   struct mallinfo nm = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
       
  3184   if (!PREACTION(m)) {
       
  3185     check_malloc_state(m);
       
  3186     if (is_initialized(m)) {
       
  3187       size_t nfree = SIZE_T_ONE; /* top always free */
       
  3188       size_t mfree = m->topsize + TOP_FOOT_SIZE;
       
  3189       size_t sum = mfree;
       
  3190       msegmentptr s = &m->seg;
       
  3191       while (s != 0) {
       
  3192         mchunkptr q = align_as_chunk(s->base);
       
  3193         while (segment_holds(s, q) &&
       
  3194                q != m->top && q->head != FENCEPOST_HEAD) {
       
  3195           size_t sz = chunksize(q);
       
  3196           sum += sz;
       
  3197           if (!cinuse(q)) {
       
  3198             mfree += sz;
       
  3199             ++nfree;
       
  3200           }
       
  3201           q = next_chunk(q);
       
  3202         }
       
  3203         s = s->next;
       
  3204       }
       
  3205 
       
  3206       nm.arena    = sum;
       
  3207       nm.ordblks  = nfree;
       
  3208       nm.hblkhd   = m->footprint - sum;
       
  3209       nm.usmblks  = m->max_footprint;
       
  3210       nm.uordblks = m->footprint - mfree;
       
  3211       nm.fordblks = mfree;
       
  3212       nm.keepcost = m->topsize;
       
  3213     }
       
  3214 
       
  3215     POSTACTION(m);
       
  3216   }
       
  3217   return nm;
       
  3218 }
       
  3219 #endif /* !NO_MALLINFO */
       
  3220 
       
  3221 static void internal_malloc_stats(mstate m) {
       
  3222   if (!PREACTION(m)) {
       
  3223     size_t maxfp = 0;
       
  3224     size_t fp = 0;
       
  3225     size_t used = 0;
       
  3226     check_malloc_state(m);
       
  3227     if (is_initialized(m)) {
       
  3228       msegmentptr s = &m->seg;
       
  3229       maxfp = m->max_footprint;
       
  3230       fp = m->footprint;
       
  3231       used = fp - (m->topsize + TOP_FOOT_SIZE);
       
  3232 
       
  3233       while (s != 0) {
       
  3234         mchunkptr q = align_as_chunk(s->base);
       
  3235         while (segment_holds(s, q) &&
       
  3236                q != m->top && q->head != FENCEPOST_HEAD) {
       
  3237           if (!cinuse(q))
       
  3238             used -= chunksize(q);
       
  3239           q = next_chunk(q);
       
  3240         }
       
  3241         s = s->next;
       
  3242       }
       
  3243     }
       
  3244 
       
  3245     fprintf(stderr, "max system bytes = %10lu\n", (unsigned long)(maxfp));
       
  3246     fprintf(stderr, "system bytes     = %10lu\n", (unsigned long)(fp));
       
  3247     fprintf(stderr, "in use bytes     = %10lu\n", (unsigned long)(used));
       
  3248 
       
  3249     POSTACTION(m);
       
  3250   }
       
  3251 }
       
  3252 
       
  3253 /* ----------------------- Operations on smallbins ----------------------- */
       
  3254 
       
  3255 /*
       
  3256   Various forms of linking and unlinking are defined as macros.  Even
       
  3257   the ones for trees, which are very long but have very short typical
       
  3258   paths.  This is ugly but reduces reliance on inlining support of
       
  3259   compilers.
       
  3260 */
       
  3261 
       
  3262 /* Link a free chunk into a smallbin  */
       
  3263 #define insert_small_chunk(M, P, S) {\
       
  3264   bindex_t I  = small_index(S);\
       
  3265   mchunkptr B = smallbin_at(M, I);\
       
  3266   mchunkptr F = B;\
       
  3267   assert(S >= MIN_CHUNK_SIZE);\
       
  3268   if (!smallmap_is_marked(M, I))\
       
  3269     mark_smallmap(M, I);\
       
  3270   else if (RTCHECK(ok_address(M, B->fd)))\
       
  3271     F = B->fd;\
       
  3272   else {\
       
  3273     CORRUPTION_ERROR_ACTION(M);\
       
  3274   }\
       
  3275   B->fd = P;\
       
  3276   F->bk = P;\
       
  3277   P->fd = F;\
       
  3278   P->bk = B;\
       
  3279 }
       
  3280 
       
  3281 /* Unlink a chunk from a smallbin  */
       
  3282 #define unlink_small_chunk(M, P, S) {\
       
  3283   mchunkptr F = P->fd;\
       
  3284   mchunkptr B = P->bk;\
       
  3285   bindex_t I = small_index(S);\
       
  3286   assert(P != B);\
       
  3287   assert(P != F);\
       
  3288   assert(chunksize(P) == small_index2size(I));\
       
  3289   if (F == B)\
       
  3290     clear_smallmap(M, I);\
       
  3291   else if (RTCHECK((F == smallbin_at(M,I) || ok_address(M, F)) &&\
       
  3292                    (B == smallbin_at(M,I) || ok_address(M, B)))) {\
       
  3293     F->bk = B;\
       
  3294     B->fd = F;\
       
  3295   }\
       
  3296   else {\
       
  3297     CORRUPTION_ERROR_ACTION(M);\
       
  3298   }\
       
  3299 }
       
  3300 
       
  3301 /* Unlink the first chunk from a smallbin */
       
  3302 #define unlink_first_small_chunk(M, B, P, I) {\
       
  3303   mchunkptr F = P->fd;\
       
  3304   assert(P != B);\
       
  3305   assert(P != F);\
       
  3306   assert(chunksize(P) == small_index2size(I));\
       
  3307   if (B == F)\
       
  3308     clear_smallmap(M, I);\
       
  3309   else if (RTCHECK(ok_address(M, F))) {\
       
  3310     B->fd = F;\
       
  3311     F->bk = B;\
       
  3312   }\
       
  3313   else {\
       
  3314     CORRUPTION_ERROR_ACTION(M);\
       
  3315   }\
       
  3316 }
       
  3317 
       
  3318 /* Replace dv node, binning the old one */
       
  3319 /* Used only when dvsize known to be small */
       
  3320 #define replace_dv(M, P, S) {\
       
  3321   size_t DVS = M->dvsize;\
       
  3322   if (DVS != 0) {\
       
  3323     mchunkptr DV = M->dv;\
       
  3324     assert(is_small(DVS));\
       
  3325     insert_small_chunk(M, DV, DVS);\
       
  3326   }\
       
  3327   M->dvsize = S;\
       
  3328   M->dv = P;\
       
  3329 }
       
  3330 
       
  3331 /* ------------------------- Operations on trees ------------------------- */
       
  3332 
       
  3333 /* Insert chunk into tree */
       
  3334 #define insert_large_chunk(M, X, S) {\
       
  3335   tbinptr* H;\
       
  3336   bindex_t I;\
       
  3337   compute_tree_index(S, I);\
       
  3338   H = treebin_at(M, I);\
       
  3339   X->index = I;\
       
  3340   X->child[0] = X->child[1] = 0;\
       
  3341   if (!treemap_is_marked(M, I)) {\
       
  3342     mark_treemap(M, I);\
       
  3343     *H = X;\
       
  3344     X->parent = (tchunkptr)H;\
       
  3345     X->fd = X->bk = X;\
       
  3346   }\
       
  3347   else {\
       
  3348     tchunkptr T = *H;\
       
  3349     size_t K = S << leftshift_for_tree_index(I);\
       
  3350     for (;;) {\
       
  3351       if (chunksize(T) != S) {\
       
  3352         tchunkptr* C = &(T->child[(K >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]);\
       
  3353         K <<= 1;\
       
  3354         if (*C != 0)\
       
  3355           T = *C;\
       
  3356         else if (RTCHECK(ok_address(M, C))) {\
       
  3357           *C = X;\
       
  3358           X->parent = T;\
       
  3359           X->fd = X->bk = X;\
       
  3360           break;\
       
  3361         }\
       
  3362         else {\
       
  3363           CORRUPTION_ERROR_ACTION(M);\
       
  3364           break;\
       
  3365         }\
       
  3366       }\
       
  3367       else {\
       
  3368         tchunkptr F = T->fd;\
       
  3369         if (RTCHECK(ok_address(M, T) && ok_address(M, F))) {\
       
  3370           T->fd = F->bk = X;\
       
  3371           X->fd = F;\
       
  3372           X->bk = T;\
       
  3373           X->parent = 0;\
       
  3374           break;\
       
  3375         }\
       
  3376         else {\
       
  3377           CORRUPTION_ERROR_ACTION(M);\
       
  3378           break;\
       
  3379         }\
       
  3380       }\
       
  3381     }\
       
  3382   }\
       
  3383 }
       
  3384 
       
  3385 /*
       
  3386   Unlink steps:
       
  3387 
       
  3388   1. If x is a chained node, unlink it from its same-sized fd/bk links
       
  3389      and choose its bk node as its replacement.
       
  3390   2. If x was the last node of its size, but not a leaf node, it must
       
  3391      be replaced with a leaf node (not merely one with an open left or
       
  3392      right), to make sure that lefts and rights of descendents
       
  3393      correspond properly to bit masks.  We use the rightmost descendent
       
  3394      of x.  We could use any other leaf, but this is easy to locate and
       
  3395      tends to counteract removal of leftmosts elsewhere, and so keeps
       
  3396      paths shorter than minimally guaranteed.  This doesn't loop much
       
  3397      because on average a node in a tree is near the bottom.
       
  3398   3. If x is the base of a chain (i.e., has parent links) relink
       
  3399      x's parent and children to x's replacement (or null if none).
       
  3400 */
       
  3401 
       
  3402 #define unlink_large_chunk(M, X) {\
       
  3403   tchunkptr XP = X->parent;\
       
  3404   tchunkptr R;\
       
  3405   if (X->bk != X) {\
       
  3406     tchunkptr F = X->fd;\
       
  3407     R = X->bk;\
       
  3408     if (RTCHECK(ok_address(M, F))) {\
       
  3409       F->bk = R;\
       
  3410       R->fd = F;\
       
  3411     }\
       
  3412     else {\
       
  3413       CORRUPTION_ERROR_ACTION(M);\
       
  3414     }\
       
  3415   }\
       
  3416   else {\
       
  3417     tchunkptr* RP;\
       
  3418     if (((R = *(RP = &(X->child[1]))) != 0) ||\
       
  3419         ((R = *(RP = &(X->child[0]))) != 0)) {\
       
  3420       tchunkptr* CP;\
       
  3421       while ((*(CP = &(R->child[1])) != 0) ||\
       
  3422              (*(CP = &(R->child[0])) != 0)) {\
       
  3423         R = *(RP = CP);\
       
  3424       }\
       
  3425       if (RTCHECK(ok_address(M, RP)))\
       
  3426         *RP = 0;\
       
  3427       else {\
       
  3428         CORRUPTION_ERROR_ACTION(M);\
       
  3429       }\
       
  3430     }\
       
  3431   }\
       
  3432   if (XP != 0) {\
       
  3433     tbinptr* H = treebin_at(M, X->index);\
       
  3434     if (X == *H) {\
       
  3435       if ((*H = R) == 0) \
       
  3436         clear_treemap(M, X->index);\
       
  3437     }\
       
  3438     else if (RTCHECK(ok_address(M, XP))) {\
       
  3439       if (XP->child[0] == X) \
       
  3440         XP->child[0] = R;\
       
  3441       else \
       
  3442         XP->child[1] = R;\
       
  3443     }\
       
  3444     else\
       
  3445       CORRUPTION_ERROR_ACTION(M);\
       
  3446     if (R != 0) {\
       
  3447       if (RTCHECK(ok_address(M, R))) {\
       
  3448         tchunkptr C0, C1;\
       
  3449         R->parent = XP;\
       
  3450         if ((C0 = X->child[0]) != 0) {\
       
  3451           if (RTCHECK(ok_address(M, C0))) {\
       
  3452             R->child[0] = C0;\
       
  3453             C0->parent = R;\
       
  3454           }\
       
  3455           else\
       
  3456             CORRUPTION_ERROR_ACTION(M);\
       
  3457         }\
       
  3458         if ((C1 = X->child[1]) != 0) {\
       
  3459           if (RTCHECK(ok_address(M, C1))) {\
       
  3460             R->child[1] = C1;\
       
  3461             C1->parent = R;\
       
  3462           }\
       
  3463           else\
       
  3464             CORRUPTION_ERROR_ACTION(M);\
       
  3465         }\
       
  3466       }\
       
  3467       else\
       
  3468         CORRUPTION_ERROR_ACTION(M);\
       
  3469     }\
       
  3470   }\
       
  3471 }
       
  3472 
       
  3473 /* Relays to large vs small bin operations */
       
  3474 
       
  3475 #define insert_chunk(M, P, S)\
       
  3476   if (is_small(S)) insert_small_chunk(M, P, S)\
       
  3477   else { tchunkptr TP = (tchunkptr)(P); insert_large_chunk(M, TP, S); }
       
  3478 
       
  3479 #define unlink_chunk(M, P, S)\
       
  3480   if (is_small(S)) unlink_small_chunk(M, P, S)\
       
  3481   else { tchunkptr TP = (tchunkptr)(P); unlink_large_chunk(M, TP); }
       
  3482 
       
  3483 
       
  3484 /* Relays to internal calls to malloc/free from realloc, memalign etc */
       
  3485 
       
  3486 #if ONLY_MSPACES
       
  3487 #define internal_malloc(m, b) mspace_malloc(m, b)
       
  3488 #define internal_free(m, mem) mspace_free(m,mem);
       
  3489 #else /* ONLY_MSPACES */
       
  3490 #if MSPACES
       
  3491 #define internal_malloc(m, b)\
       
  3492    (m == gm)? dlmalloc(b) : mspace_malloc(m, b)
       
  3493 #define internal_free(m, mem)\
       
  3494    if (m == gm) dlfree(mem); else mspace_free(m,mem);
       
  3495 #else /* MSPACES */
       
  3496 #define internal_malloc(m, b) dlmalloc(b)
       
  3497 #define internal_free(m, mem) dlfree(mem)
       
  3498 #endif /* MSPACES */
       
  3499 #endif /* ONLY_MSPACES */
       
  3500 
       
  3501 /* -----------------------  Direct-mmapping chunks ----------------------- */
       
  3502 
       
  3503 /*
       
  3504   Directly mmapped chunks are set up with an offset to the start of
       
  3505   the mmapped region stored in the prev_foot field of the chunk. This
       
  3506   allows reconstruction of the required argument to MUNMAP when freed,
       
  3507   and also allows adjustment of the returned chunk to meet alignment
       
  3508   requirements (especially in memalign).  There is also enough space
       
  3509   allocated to hold a fake next chunk of size SIZE_T_SIZE to maintain
       
  3510   the PINUSE bit so frees can be checked.
       
  3511 */
       
  3512 
       
  3513 /* Malloc using mmap */
       
  3514 static void* mmap_alloc(mstate m, size_t nb) {
       
  3515   size_t mmsize = mmap_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK);
       
  3516   if (mmsize > nb) {     /* Check for wrap around 0 */
       
  3517     char* mm = (char*)(DIRECT_MMAP(mmsize));
       
  3518     if (mm != CMFAIL) {
       
  3519       size_t offset = align_offset(chunk2mem(mm));
       
  3520       size_t psize = mmsize - offset - MMAP_FOOT_PAD;
       
  3521       mchunkptr p = (mchunkptr)(mm + offset);
       
  3522       p->prev_foot = offset | IS_MMAPPED_BIT;
       
  3523       (p)->head = (psize|CINUSE_BIT);
       
  3524       mark_inuse_foot(m, p, psize);
       
  3525       chunk_plus_offset(p, psize)->head = FENCEPOST_HEAD;
       
  3526       chunk_plus_offset(p, psize+SIZE_T_SIZE)->head = 0;
       
  3527 
       
  3528       if (mm < m->least_addr)
       
  3529         m->least_addr = mm;
       
  3530       if ((m->footprint += mmsize) > m->max_footprint)
       
  3531         m->max_footprint = m->footprint;
       
  3532       assert(is_aligned(chunk2mem(p)));
       
  3533       check_mmapped_chunk(m, p);
       
  3534       return chunk2mem(p);
       
  3535     }
       
  3536   }
       
  3537   return 0;
       
  3538 }
       
  3539 
       
  3540 /* Realloc using mmap */
       
  3541 static mchunkptr mmap_resize(mstate m, mchunkptr oldp, size_t nb) {
       
  3542   size_t oldsize = chunksize(oldp);
       
  3543   if (is_small(nb)) /* Can't shrink mmap regions below small size */
       
  3544     return 0;
       
  3545   /* Keep old chunk if big enough but not too big */
       
  3546   if (oldsize >= nb + SIZE_T_SIZE &&
       
  3547       (oldsize - nb) <= (mparams.granularity << 1))
       
  3548     return oldp;
       
  3549   else {
       
  3550     size_t offset = oldp->prev_foot & ~IS_MMAPPED_BIT;
       
  3551     size_t oldmmsize = oldsize + offset + MMAP_FOOT_PAD;
       
  3552     size_t newmmsize = mmap_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK);
       
  3553     char* cp = (char*)CALL_MREMAP((char*)oldp - offset,
       
  3554                                   oldmmsize, newmmsize, 1);
       
  3555     if (cp != CMFAIL) {
       
  3556       mchunkptr newp = (mchunkptr)(cp + offset);
       
  3557       size_t psize = newmmsize - offset - MMAP_FOOT_PAD;
       
  3558       newp->head = (psize|CINUSE_BIT);
       
  3559       mark_inuse_foot(m, newp, psize);
       
  3560       chunk_plus_offset(newp, psize)->head = FENCEPOST_HEAD;
       
  3561       chunk_plus_offset(newp, psize+SIZE_T_SIZE)->head = 0;
       
  3562 
       
  3563       if (cp < m->least_addr)
       
  3564         m->least_addr = cp;
       
  3565       if ((m->footprint += newmmsize - oldmmsize) > m->max_footprint)
       
  3566         m->max_footprint = m->footprint;
       
  3567       check_mmapped_chunk(m, newp);
       
  3568       return newp;
       
  3569     }
       
  3570   }
       
  3571   return 0;
       
  3572 }
       
  3573 
       
  3574 /* -------------------------- mspace management -------------------------- */
       
  3575 
       
  3576 /* Initialize top chunk and its size */
       
  3577 static void init_top(mstate m, mchunkptr p, size_t psize) {
       
  3578   /* Ensure alignment */
       
  3579   size_t offset = align_offset(chunk2mem(p));
       
  3580   p = (mchunkptr)((char*)p + offset);
       
  3581   psize -= offset;
       
  3582 
       
  3583   m->top = p;
       
  3584   m->topsize = psize;
       
  3585   p->head = psize | PINUSE_BIT;
       
  3586   /* set size of fake trailing chunk holding overhead space only once */
       
  3587   chunk_plus_offset(p, psize)->head = TOP_FOOT_SIZE;
       
  3588   m->trim_check = mparams.trim_threshold; /* reset on each update */
       
  3589 }
       
  3590 
       
  3591 /* Initialize bins for a new mstate that is otherwise zeroed out */
       
  3592 static void init_bins(mstate m) {
       
  3593   /* Establish circular links for smallbins */
       
  3594   bindex_t i;
       
  3595   for (i = 0; i < NSMALLBINS; ++i) {
       
  3596     sbinptr bin = smallbin_at(m,i);
       
  3597     bin->fd = bin->bk = bin;
       
  3598   }
       
  3599 }
       
  3600 
       
  3601 #if PROCEED_ON_ERROR
       
  3602 
       
  3603 /* default corruption action */
       
  3604 static void reset_on_error(mstate m) {
       
  3605   int i;
       
  3606   ++malloc_corruption_error_count;
       
  3607   /* Reinitialize fields to forget about all memory */
       
  3608   m->smallbins = m->treebins = 0;
       
  3609   m->dvsize = m->topsize = 0;
       
  3610   m->seg.base = 0;
       
  3611   m->seg.size = 0;
       
  3612   m->seg.next = 0;
       
  3613   m->top = m->dv = 0;
       
  3614   for (i = 0; i < NTREEBINS; ++i)
       
  3615     *treebin_at(m, i) = 0;
       
  3616   init_bins(m);
       
  3617 }
       
  3618 #endif /* PROCEED_ON_ERROR */
       
  3619 
       
  3620 /* Allocate chunk and prepend remainder with chunk in successor base. */
       
  3621 static void* prepend_alloc(mstate m, char* newbase, char* oldbase,
       
  3622                            size_t nb) {
       
  3623   mchunkptr p = align_as_chunk(newbase);
       
  3624   mchunkptr oldfirst = align_as_chunk(oldbase);
       
  3625   size_t psize = (char*)oldfirst - (char*)p;
       
  3626   mchunkptr q = chunk_plus_offset(p, nb);
       
  3627   size_t qsize = psize - nb;
       
  3628   set_size_and_pinuse_of_inuse_chunk(m, p, nb);
       
  3629 
       
  3630   assert((char*)oldfirst > (char*)q);
       
  3631   assert(pinuse(oldfirst));
       
  3632   assert(qsize >= MIN_CHUNK_SIZE);
       
  3633 
       
  3634   /* consolidate remainder with first chunk of old base */
       
  3635   if (oldfirst == m->top) {
       
  3636     size_t tsize = m->topsize += qsize;
       
  3637     m->top = q;
       
  3638     q->head = tsize | PINUSE_BIT;
       
  3639     check_top_chunk(m, q);
       
  3640   }
       
  3641   else if (oldfirst == m->dv) {
       
  3642     size_t dsize = m->dvsize += qsize;
       
  3643     m->dv = q;
       
  3644     set_size_and_pinuse_of_free_chunk(q, dsize);
       
  3645   }
       
  3646   else {
       
  3647     if (!cinuse(oldfirst)) {
       
  3648       size_t nsize = chunksize(oldfirst);
       
  3649       unlink_chunk(m, oldfirst, nsize);
       
  3650       oldfirst = chunk_plus_offset(oldfirst, nsize);
       
  3651       qsize += nsize;
       
  3652     }
       
  3653     set_free_with_pinuse(q, qsize, oldfirst);
       
  3654     insert_chunk(m, q, qsize);
       
  3655     check_free_chunk(m, q);
       
  3656   }
       
  3657 
       
  3658   check_malloced_chunk(m, chunk2mem(p), nb);
       
  3659   return chunk2mem(p);
       
  3660 }
       
  3661 
       
  3662 /* Add a segment to hold a new noncontiguous region */
       
  3663 static void add_segment(mstate m, char* tbase, size_t tsize, flag_t mmapped) {
       
  3664   /* Determine locations and sizes of segment, fenceposts, old top */
       
  3665   char* old_top = (char*)m->top;
       
  3666   msegmentptr oldsp = segment_holding(m, old_top);
       
  3667   char* old_end = oldsp->base + oldsp->size;
       
  3668   size_t ssize = pad_request(sizeof(struct malloc_segment));
       
  3669   char* rawsp = old_end - (ssize + FOUR_SIZE_T_SIZES + CHUNK_ALIGN_MASK);
       
  3670   size_t offset = align_offset(chunk2mem(rawsp));
       
  3671   char* asp = rawsp + offset;
       
  3672   char* csp = (asp < (old_top + MIN_CHUNK_SIZE))? old_top : asp;
       
  3673   mchunkptr sp = (mchunkptr)csp;
       
  3674   msegmentptr ss = (msegmentptr)(chunk2mem(sp));
       
  3675   mchunkptr tnext = chunk_plus_offset(sp, ssize);
       
  3676   mchunkptr p = tnext;
       
  3677   int nfences = 0;
       
  3678 
       
  3679   /* reset top to new space */
       
  3680   init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE);
       
  3681 
       
  3682   /* Set up segment record */
       
  3683   assert(is_aligned(ss));
       
  3684   set_size_and_pinuse_of_inuse_chunk(m, sp, ssize);
       
  3685   *ss = m->seg; /* Push current record */
       
  3686   m->seg.base = tbase;
       
  3687   m->seg.size = tsize;
       
  3688   m->seg.sflags = mmapped;
       
  3689   m->seg.next = ss;
       
  3690 
       
  3691   /* Insert trailing fenceposts */
       
  3692   for (;;) {
       
  3693     mchunkptr nextp = chunk_plus_offset(p, SIZE_T_SIZE);
       
  3694     p->head = FENCEPOST_HEAD;
       
  3695     ++nfences;
       
  3696     if ((char*)(&(nextp->head)) < old_end)
       
  3697       p = nextp;
       
  3698     else
       
  3699       break;
       
  3700   }
       
  3701   assert(nfences >= 2);
       
  3702 
       
  3703   /* Insert the rest of old top into a bin as an ordinary free chunk */
       
  3704   if (csp != old_top) {
       
  3705     mchunkptr q = (mchunkptr)old_top;
       
  3706     size_t psize = csp - old_top;
       
  3707     mchunkptr tn = chunk_plus_offset(q, psize);
       
  3708     set_free_with_pinuse(q, psize, tn);
       
  3709     insert_chunk(m, q, psize);
       
  3710   }
       
  3711 
       
  3712   check_top_chunk(m, m->top);
       
  3713 }
       
  3714 
       
  3715 /* -------------------------- System allocation -------------------------- */
       
  3716 
       
  3717 /* Get memory from system using MORECORE or MMAP */
       
  3718 static void* sys_alloc(mstate m, size_t nb) {
       
  3719   char* tbase = CMFAIL;
       
  3720   size_t tsize = 0;
       
  3721   flag_t mmap_flag = 0;
       
  3722 
       
  3723   init_mparams();
       
  3724 
       
  3725   /* Directly map large chunks */
       
  3726   if (use_mmap(m) && nb >= mparams.mmap_threshold) {
       
  3727     void* mem = mmap_alloc(m, nb);
       
  3728     if (mem != 0)
       
  3729       return mem;
       
  3730   }
       
  3731 
       
  3732   /*
       
  3733     Try getting memory in any of three ways (in most-preferred to
       
  3734     least-preferred order):
       
  3735     1. A call to MORECORE that can normally contiguously extend memory.
       
  3736        (disabled if not MORECORE_CONTIGUOUS or not HAVE_MORECORE or
       
  3737        or main space is mmapped or a previous contiguous call failed)
       
  3738     2. A call to MMAP new space (disabled if not HAVE_MMAP).
       
  3739        Note that under the default settings, if MORECORE is unable to
       
  3740        fulfill a request, and HAVE_MMAP is true, then mmap is
       
  3741        used as a noncontiguous system allocator. This is a useful backup
       
  3742        strategy for systems with holes in address spaces -- in this case
       
  3743        sbrk cannot contiguously expand the heap, but mmap may be able to
       
  3744        find space.
       
  3745     3. A call to MORECORE that cannot usually contiguously extend memory.
       
  3746        (disabled if not HAVE_MORECORE)
       
  3747   */
       
  3748 
       
  3749   if (MORECORE_CONTIGUOUS && !use_noncontiguous(m)) {
       
  3750     char* br = CMFAIL;
       
  3751     msegmentptr ss = (m->top == 0)? 0 : segment_holding(m, (char*)m->top);
       
  3752     size_t asize = 0;
       
  3753     ACQUIRE_MORECORE_LOCK();
       
  3754 
       
  3755     if (ss == 0) {  /* First time through or recovery */
       
  3756       char* base = (char*)CALL_MORECORE(0);
       
  3757       if (base != CMFAIL) {
       
  3758         asize = granularity_align(nb + TOP_FOOT_SIZE + SIZE_T_ONE);
       
  3759         /* Adjust to end on a page boundary */
       
  3760         if (!is_page_aligned(base))
       
  3761           asize += (page_align((size_t)base) - (size_t)base);
       
  3762         /* Can't call MORECORE if size is negative when treated as signed */
       
  3763         if (asize < HALF_MAX_SIZE_T &&
       
  3764             (br = (char*)(CALL_MORECORE(asize))) == base) {
       
  3765           tbase = base;
       
  3766           tsize = asize;
       
  3767         }
       
  3768       }
       
  3769     }
       
  3770     else {
       
  3771       /* Subtract out existing available top space from MORECORE request. */
       
  3772       asize = granularity_align(nb - m->topsize + TOP_FOOT_SIZE + SIZE_T_ONE);
       
  3773       /* Use mem here only if it did continuously extend old space */
       
  3774       if (asize < HALF_MAX_SIZE_T &&
       
  3775           (br = (char*)(CALL_MORECORE(asize))) == ss->base+ss->size) {
       
  3776         tbase = br;
       
  3777         tsize = asize;
       
  3778       }
       
  3779     }
       
  3780 
       
  3781     if (tbase == CMFAIL) {    /* Cope with partial failure */
       
  3782       if (br != CMFAIL) {    /* Try to use/extend the space we did get */
       
  3783         if (asize < HALF_MAX_SIZE_T &&
       
  3784             asize < nb + TOP_FOOT_SIZE + SIZE_T_ONE) {
       
  3785           size_t esize = granularity_align(nb + TOP_FOOT_SIZE + SIZE_T_ONE - asize);
       
  3786           if (esize < HALF_MAX_SIZE_T) {
       
  3787             char* end = (char*)CALL_MORECORE(esize);
       
  3788             if (end != CMFAIL)
       
  3789               asize += esize;
       
  3790             else {            /* Can't use; try to release */
       
  3791               (void) CALL_MORECORE(-asize);
       
  3792               br = CMFAIL;
       
  3793             }
       
  3794           }
       
  3795         }
       
  3796       }
       
  3797       if (br != CMFAIL) {    /* Use the space we did get */
       
  3798         tbase = br;
       
  3799         tsize = asize;
       
  3800       }
       
  3801       else
       
  3802         disable_contiguous(m); /* Don't try contiguous path in the future */
       
  3803     }
       
  3804 
       
  3805     RELEASE_MORECORE_LOCK();
       
  3806   }
       
  3807 
       
  3808   if (HAVE_MMAP && tbase == CMFAIL) {  /* Try MMAP */
       
  3809     size_t req = nb + TOP_FOOT_SIZE + SIZE_T_ONE;
       
  3810     size_t rsize = granularity_align(req);
       
  3811     if (rsize > nb) { /* Fail if wraps around zero */
       
  3812       char* mp = (char*)(CALL_MMAP(rsize));
       
  3813       if (mp != CMFAIL) {
       
  3814         tbase = mp;
       
  3815         tsize = rsize;
       
  3816         mmap_flag = IS_MMAPPED_BIT;
       
  3817       }
       
  3818     }
       
  3819   }
       
  3820 
       
  3821   if (HAVE_MORECORE && tbase == CMFAIL) { /* Try noncontiguous MORECORE */
       
  3822     size_t asize = granularity_align(nb + TOP_FOOT_SIZE + SIZE_T_ONE);
       
  3823     if (asize < HALF_MAX_SIZE_T) {
       
  3824       char* br = CMFAIL;
       
  3825       char* end = CMFAIL;
       
  3826       ACQUIRE_MORECORE_LOCK();
       
  3827       br = (char*)(CALL_MORECORE(asize));
       
  3828       end = (char*)(CALL_MORECORE(0));
       
  3829       RELEASE_MORECORE_LOCK();
       
  3830       if (br != CMFAIL && end != CMFAIL && br < end) {
       
  3831         size_t ssize = end - br;
       
  3832         if (ssize > nb + TOP_FOOT_SIZE) {
       
  3833           tbase = br;
       
  3834           tsize = ssize;
       
  3835         }
       
  3836       }
       
  3837     }
       
  3838   }
       
  3839 
       
  3840   if (tbase != CMFAIL) {
       
  3841 
       
  3842     if ((m->footprint += tsize) > m->max_footprint)
       
  3843       m->max_footprint = m->footprint;
       
  3844 
       
  3845     if (!is_initialized(m)) { /* first-time initialization */
       
  3846       m->seg.base = m->least_addr = tbase;
       
  3847       m->seg.size = tsize;
       
  3848       m->seg.sflags = mmap_flag;
       
  3849       m->magic = mparams.magic;
       
  3850       m->release_checks = MAX_RELEASE_CHECK_RATE;
       
  3851       init_bins(m);
       
  3852 #if !ONLY_MSPACES
       
  3853       if (is_global(m))
       
  3854         init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE);
       
  3855       else
       
  3856 #endif
       
  3857       {
       
  3858         /* Offset top by embedded malloc_state */
       
  3859         mchunkptr mn = next_chunk(mem2chunk(m));
       
  3860         init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) -TOP_FOOT_SIZE);
       
  3861       }
       
  3862     }
       
  3863 
       
  3864     else {
       
  3865       /* Try to merge with an existing segment */
       
  3866       msegmentptr sp = &m->seg;
       
  3867       /* Only consider most recent segment if traversal suppressed */
       
  3868       while (sp != 0 && tbase != sp->base + sp->size)
       
  3869         sp = (NO_SEGMENT_TRAVERSAL) ? 0 : sp->next;
       
  3870       if (sp != 0 &&
       
  3871           !is_extern_segment(sp) &&
       
  3872           (sp->sflags & IS_MMAPPED_BIT) == mmap_flag &&
       
  3873           segment_holds(sp, m->top)) { /* append */
       
  3874         sp->size += tsize;
       
  3875         init_top(m, m->top, m->topsize + tsize);
       
  3876       }
       
  3877       else {
       
  3878         if (tbase < m->least_addr)
       
  3879           m->least_addr = tbase;
       
  3880         sp = &m->seg;
       
  3881         while (sp != 0 && sp->base != tbase + tsize)
       
  3882           sp = (NO_SEGMENT_TRAVERSAL) ? 0 : sp->next;
       
  3883         if (sp != 0 &&
       
  3884             !is_extern_segment(sp) &&
       
  3885             (sp->sflags & IS_MMAPPED_BIT) == mmap_flag) {
       
  3886           char* oldbase = sp->base;
       
  3887           sp->base = tbase;
       
  3888           sp->size += tsize;
       
  3889           return prepend_alloc(m, tbase, oldbase, nb);
       
  3890         }
       
  3891         else
       
  3892           add_segment(m, tbase, tsize, mmap_flag);
       
  3893       }
       
  3894     }
       
  3895 
       
  3896     if (nb < m->topsize) { /* Allocate from new or extended top space */
       
  3897       size_t rsize = m->topsize -= nb;
       
  3898       mchunkptr p = m->top;
       
  3899       mchunkptr r = m->top = chunk_plus_offset(p, nb);
       
  3900       r->head = rsize | PINUSE_BIT;
       
  3901       set_size_and_pinuse_of_inuse_chunk(m, p, nb);
       
  3902       check_top_chunk(m, m->top);
       
  3903       check_malloced_chunk(m, chunk2mem(p), nb);
       
  3904       return chunk2mem(p);
       
  3905     }
       
  3906   }
       
  3907 
       
  3908   MALLOC_FAILURE_ACTION;
       
  3909   return 0;
       
  3910 }
       
  3911 
       
  3912 /* -----------------------  system deallocation -------------------------- */
       
  3913 
       
  3914 /* Unmap and unlink any mmapped segments that don't contain used chunks */
       
  3915 static size_t release_unused_segments(mstate m) {
       
  3916   size_t released = 0;
       
  3917   int nsegs = 0;
       
  3918   msegmentptr pred = &m->seg;
       
  3919   msegmentptr sp = pred->next;
       
  3920   while (sp != 0) {
       
  3921     char* base = sp->base;
       
  3922     size_t size = sp->size;
       
  3923     msegmentptr next = sp->next;
       
  3924     ++nsegs;
       
  3925     if (is_mmapped_segment(sp) && !is_extern_segment(sp)) {
       
  3926       mchunkptr p = align_as_chunk(base);
       
  3927       size_t psize = chunksize(p);
       
  3928       /* Can unmap if first chunk holds entire segment and not pinned */
       
  3929       if (!cinuse(p) && (char*)p + psize >= base + size - TOP_FOOT_SIZE) {
       
  3930         tchunkptr tp = (tchunkptr)p;
       
  3931         assert(segment_holds(sp, (char*)sp));
       
  3932         if (p == m->dv) {
       
  3933           m->dv = 0;
       
  3934           m->dvsize = 0;
       
  3935         }
       
  3936         else {
       
  3937           unlink_large_chunk(m, tp);
       
  3938         }
       
  3939         if (CALL_MUNMAP(base, size) == 0) {
       
  3940           released += size;
       
  3941           m->footprint -= size;
       
  3942           /* unlink obsoleted record */
       
  3943           sp = pred;
       
  3944           sp->next = next;
       
  3945         }
       
  3946         else { /* back out if cannot unmap */
       
  3947           insert_large_chunk(m, tp, psize);
       
  3948         }
       
  3949       }
       
  3950     }
       
  3951     if (NO_SEGMENT_TRAVERSAL) /* scan only first segment */
       
  3952       break;
       
  3953     pred = sp;
       
  3954     sp = next;
       
  3955   }
       
  3956   /* Reset check counter */
       
  3957   m->release_checks = ((nsegs > MAX_RELEASE_CHECK_RATE)?
       
  3958                        nsegs : MAX_RELEASE_CHECK_RATE);
       
  3959   return released;
       
  3960 }
       
  3961 
       
  3962 static int sys_trim(mstate m, size_t pad) {
       
  3963   size_t released = 0;
       
  3964   if (pad < MAX_REQUEST && is_initialized(m)) {
       
  3965     pad += TOP_FOOT_SIZE; /* ensure enough room for segment overhead */
       
  3966 
       
  3967     if (m->topsize > pad) {
       
  3968       /* Shrink top space in granularity-size units, keeping at least one */
       
  3969       size_t unit = mparams.granularity;
       
  3970       size_t extra = ((m->topsize - pad + (unit - SIZE_T_ONE)) / unit -
       
  3971                       SIZE_T_ONE) * unit;
       
  3972       msegmentptr sp = segment_holding(m, (char*)m->top);
       
  3973 
       
  3974       if (!is_extern_segment(sp)) {
       
  3975         if (is_mmapped_segment(sp)) {
       
  3976           if (HAVE_MMAP &&
       
  3977               sp->size >= extra &&
       
  3978               !has_segment_link(m, sp)) { /* can't shrink if pinned */
       
  3979             size_t newsize = sp->size - extra;
       
  3980             /* Prefer mremap, fall back to munmap */
       
  3981             if ((CALL_MREMAP(sp->base, sp->size, newsize, 0) != MFAIL) ||
       
  3982                 (CALL_MUNMAP(sp->base + newsize, extra) == 0)) {
       
  3983               released = extra;
       
  3984             }
       
  3985           }
       
  3986         }
       
  3987         else if (HAVE_MORECORE) {
       
  3988           if (extra >= HALF_MAX_SIZE_T) /* Avoid wrapping negative */
       
  3989             extra = (HALF_MAX_SIZE_T) + SIZE_T_ONE - unit;
       
  3990           ACQUIRE_MORECORE_LOCK();
       
  3991           {
       
  3992             /* Make sure end of memory is where we last set it. */
       
  3993             char* old_br = (char*)(CALL_MORECORE(0));
       
  3994             if (old_br == sp->base + sp->size) {
       
  3995               char* rel_br = (char*)(CALL_MORECORE(-extra));
       
  3996               char* new_br = (char*)(CALL_MORECORE(0));
       
  3997               if (rel_br != CMFAIL && new_br < old_br)
       
  3998                 released = old_br - new_br;
       
  3999             }
       
  4000           }
       
  4001           RELEASE_MORECORE_LOCK();
       
  4002         }
       
  4003       }
       
  4004 
       
  4005       if (released != 0) {
       
  4006         sp->size -= released;
       
  4007         m->footprint -= released;
       
  4008         init_top(m, m->top, m->topsize - released);
       
  4009         check_top_chunk(m, m->top);
       
  4010       }
       
  4011     }
       
  4012 
       
  4013     /* Unmap any unused mmapped segments */
       
  4014     if (HAVE_MMAP)
       
  4015       released += release_unused_segments(m);
       
  4016 
       
  4017     /* On failure, disable autotrim to avoid repeated failed future calls */
       
  4018     if (released == 0 && m->topsize > m->trim_check)
       
  4019       m->trim_check = MAX_SIZE_T;
       
  4020   }
       
  4021 
       
  4022   return (released != 0)? 1 : 0;
       
  4023 }
       
  4024 
       
  4025 /* ---------------------------- malloc support --------------------------- */
       
  4026 
       
  4027 /* allocate a large request from the best fitting chunk in a treebin */
       
  4028 static void* tmalloc_large(mstate m, size_t nb) {
       
  4029   tchunkptr v = 0;
       
  4030   size_t rsize = -nb; /* Unsigned negation */
       
  4031   tchunkptr t;
       
  4032   bindex_t idx;
       
  4033   compute_tree_index(nb, idx);
       
  4034 
       
  4035   if ((t = *treebin_at(m, idx)) != 0) {
       
  4036     /* Traverse tree for this bin looking for node with size == nb */
       
  4037     size_t sizebits = nb << leftshift_for_tree_index(idx);
       
  4038     tchunkptr rst = 0;  /* The deepest untaken right subtree */
       
  4039     for (;;) {
       
  4040       tchunkptr rt;
       
  4041       size_t trem = chunksize(t) - nb;
       
  4042       if (trem < rsize) {
       
  4043         v = t;
       
  4044         if ((rsize = trem) == 0)
       
  4045           break;
       
  4046       }
       
  4047       rt = t->child[1];
       
  4048       t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1];
       
  4049       if (rt != 0 && rt != t)
       
  4050         rst = rt;
       
  4051       if (t == 0) {
       
  4052         t = rst; /* set t to least subtree holding sizes > nb */
       
  4053         break;
       
  4054       }
       
  4055       sizebits <<= 1;
       
  4056     }
       
  4057   }
       
  4058 
       
  4059   if (t == 0 && v == 0) { /* set t to root of next non-empty treebin */
       
  4060     binmap_t leftbits = left_bits(idx2bit(idx)) & m->treemap;
       
  4061     if (leftbits != 0) {
       
  4062       bindex_t i;
       
  4063       binmap_t leastbit = least_bit(leftbits);
       
  4064       compute_bit2idx(leastbit, i);
       
  4065       t = *treebin_at(m, i);
       
  4066     }
       
  4067   }
       
  4068 
       
  4069   while (t != 0) { /* find smallest of tree or subtree */
       
  4070     size_t trem = chunksize(t) - nb;
       
  4071     if (trem < rsize) {
       
  4072       rsize = trem;
       
  4073       v = t;
       
  4074     }
       
  4075     t = leftmost_child(t);
       
  4076   }
       
  4077 
       
  4078   /*  If dv is a better fit, return 0 so malloc will use it */
       
  4079   if (v != 0 && rsize < (size_t)(m->dvsize - nb)) {
       
  4080     if (RTCHECK(ok_address(m, v))) { /* split */
       
  4081       mchunkptr r = chunk_plus_offset(v, nb);
       
  4082       assert(chunksize(v) == rsize + nb);
       
  4083       if (RTCHECK(ok_next(v, r))) {
       
  4084         unlink_large_chunk(m, v);
       
  4085         if (rsize < MIN_CHUNK_SIZE)
       
  4086           set_inuse_and_pinuse(m, v, (rsize + nb));
       
  4087         else {
       
  4088           set_size_and_pinuse_of_inuse_chunk(m, v, nb);
       
  4089           set_size_and_pinuse_of_free_chunk(r, rsize);
       
  4090           insert_chunk(m, r, rsize);
       
  4091         }
       
  4092         return chunk2mem(v);
       
  4093       }
       
  4094     }
       
  4095     CORRUPTION_ERROR_ACTION(m);
       
  4096   }
       
  4097   return 0;
       
  4098 }
       
  4099 
       
  4100 /* allocate a small request from the best fitting chunk in a treebin */
       
  4101 static void* tmalloc_small(mstate m, size_t nb) {
       
  4102   tchunkptr t, v;
       
  4103   size_t rsize;
       
  4104   bindex_t i;
       
  4105   binmap_t leastbit = least_bit(m->treemap);
       
  4106   compute_bit2idx(leastbit, i);
       
  4107 
       
  4108   v = t = *treebin_at(m, i);
       
  4109   rsize = chunksize(t) - nb;
       
  4110 
       
  4111   while ((t = leftmost_child(t)) != 0) {
       
  4112     size_t trem = chunksize(t) - nb;
       
  4113     if (trem < rsize) {
       
  4114       rsize = trem;
       
  4115       v = t;
       
  4116     }
       
  4117   }
       
  4118 
       
  4119   if (RTCHECK(ok_address(m, v))) {
       
  4120     mchunkptr r = chunk_plus_offset(v, nb);
       
  4121     assert(chunksize(v) == rsize + nb);
       
  4122     if (RTCHECK(ok_next(v, r))) {
       
  4123       unlink_large_chunk(m, v);
       
  4124       if (rsize < MIN_CHUNK_SIZE)
       
  4125         set_inuse_and_pinuse(m, v, (rsize + nb));
       
  4126       else {
       
  4127         set_size_and_pinuse_of_inuse_chunk(m, v, nb);
       
  4128         set_size_and_pinuse_of_free_chunk(r, rsize);
       
  4129         replace_dv(m, r, rsize);
       
  4130       }
       
  4131       return chunk2mem(v);
       
  4132     }
       
  4133   }
       
  4134 
       
  4135   CORRUPTION_ERROR_ACTION(m);
       
  4136   return 0;
       
  4137 }
       
  4138 
       
  4139 /* --------------------------- realloc support --------------------------- */
       
  4140 
       
  4141 static void* internal_realloc(mstate m, void* oldmem, size_t bytes) {
       
  4142   if (bytes >= MAX_REQUEST) {
       
  4143     MALLOC_FAILURE_ACTION;
       
  4144     return 0;
       
  4145   }
       
  4146   if (!PREACTION(m)) {
       
  4147     mchunkptr oldp = mem2chunk(oldmem);
       
  4148     size_t oldsize = chunksize(oldp);
       
  4149     mchunkptr next = chunk_plus_offset(oldp, oldsize);
       
  4150     mchunkptr newp = 0;
       
  4151     void* extra = 0;
       
  4152 
       
  4153     /* Try to either shrink or extend into top. Else malloc-copy-free */
       
  4154 
       
  4155     if (RTCHECK(ok_address(m, oldp) && ok_cinuse(oldp) &&
       
  4156                 ok_next(oldp, next) && ok_pinuse(next))) {
       
  4157       size_t nb = request2size(bytes);
       
  4158       if (is_mmapped(oldp))
       
  4159         newp = mmap_resize(m, oldp, nb);
       
  4160       else if (oldsize >= nb) { /* already big enough */
       
  4161         size_t rsize = oldsize - nb;
       
  4162         newp = oldp;
       
  4163         if (rsize >= MIN_CHUNK_SIZE) {
       
  4164           mchunkptr remainder = chunk_plus_offset(newp, nb);
       
  4165           set_inuse(m, newp, nb);
       
  4166           set_inuse(m, remainder, rsize);
       
  4167           extra = chunk2mem(remainder);
       
  4168         }
       
  4169       }
       
  4170       else if (next == m->top && oldsize + m->topsize > nb) {
       
  4171         /* Expand into top */
       
  4172         size_t newsize = oldsize + m->topsize;
       
  4173         size_t newtopsize = newsize - nb;
       
  4174         mchunkptr newtop = chunk_plus_offset(oldp, nb);
       
  4175         set_inuse(m, oldp, nb);
       
  4176         newtop->head = newtopsize |PINUSE_BIT;
       
  4177         m->top = newtop;
       
  4178         m->topsize = newtopsize;
       
  4179         newp = oldp;
       
  4180       }
       
  4181     }
       
  4182     else {
       
  4183       USAGE_ERROR_ACTION(m, oldmem);
       
  4184       POSTACTION(m);
       
  4185       return 0;
       
  4186     }
       
  4187 
       
  4188     POSTACTION(m);
       
  4189 
       
  4190     if (newp != 0) {
       
  4191       if (extra != 0) {
       
  4192         internal_free(m, extra);
       
  4193       }
       
  4194       check_inuse_chunk(m, newp);
       
  4195       return chunk2mem(newp);
       
  4196     }
       
  4197     else {
       
  4198       void* newmem = internal_malloc(m, bytes);
       
  4199       if (newmem != 0) {
       
  4200         size_t oc = oldsize - overhead_for(oldp);
       
  4201         memcpy(newmem, oldmem, (oc < bytes)? oc : bytes);
       
  4202         internal_free(m, oldmem);
       
  4203       }
       
  4204       return newmem;
       
  4205     }
       
  4206   }
       
  4207   return 0;
       
  4208 }
       
  4209 
       
  4210 /* --------------------------- memalign support -------------------------- */
       
  4211 
       
  4212 static void* internal_memalign(mstate m, size_t alignment, size_t bytes) {
       
  4213   if (alignment <= MALLOC_ALIGNMENT)    /* Can just use malloc */
       
  4214     return internal_malloc(m, bytes);
       
  4215   if (alignment <  MIN_CHUNK_SIZE) /* must be at least a minimum chunk size */
       
  4216     alignment = MIN_CHUNK_SIZE;
       
  4217   if ((alignment & (alignment-SIZE_T_ONE)) != 0) {/* Ensure a power of 2 */
       
  4218     size_t a = MALLOC_ALIGNMENT << 1;
       
  4219     while (a < alignment) a <<= 1;
       
  4220     alignment = a;
       
  4221   }
       
  4222 
       
  4223   if (bytes >= MAX_REQUEST - alignment) {
       
  4224     if (m != 0)  { /* Test isn't needed but avoids compiler warning */
       
  4225       MALLOC_FAILURE_ACTION;
       
  4226     }
       
  4227   }
       
  4228   else {
       
  4229     size_t nb = request2size(bytes);
       
  4230     size_t req = nb + alignment + MIN_CHUNK_SIZE - CHUNK_OVERHEAD;
       
  4231     char* mem = (char*)internal_malloc(m, req);
       
  4232     if (mem != 0) {
       
  4233       void* leader = 0;
       
  4234       void* trailer = 0;
       
  4235       mchunkptr p = mem2chunk(mem);
       
  4236 
       
  4237       if (PREACTION(m)) return 0;
       
  4238       if ((((size_t)(mem)) % alignment) != 0) { /* misaligned */
       
  4239         /*
       
  4240           Find an aligned spot inside chunk.  Since we need to give
       
  4241           back leading space in a chunk of at least MIN_CHUNK_SIZE, if
       
  4242           the first calculation places us at a spot with less than
       
  4243           MIN_CHUNK_SIZE leader, we can move to the next aligned spot.
       
  4244           We've allocated enough total room so that this is always
       
  4245           possible.
       
  4246         */
       
  4247         char* br = (char*)mem2chunk((size_t)(((size_t)(mem +
       
  4248                                                        alignment -
       
  4249                                                        SIZE_T_ONE)) &
       
  4250                                              -alignment));
       
  4251         char* pos = ((size_t)(br - (char*)(p)) >= MIN_CHUNK_SIZE)?
       
  4252           br : br+alignment;
       
  4253         mchunkptr newp = (mchunkptr)pos;
       
  4254         size_t leadsize = pos - (char*)(p);
       
  4255         size_t newsize = chunksize(p) - leadsize;
       
  4256 
       
  4257         if (is_mmapped(p)) { /* For mmapped chunks, just adjust offset */
       
  4258           newp->prev_foot = p->prev_foot + leadsize;
       
  4259           newp->head = (newsize|CINUSE_BIT);
       
  4260         }
       
  4261         else { /* Otherwise, give back leader, use the rest */
       
  4262           set_inuse(m, newp, newsize);
       
  4263           set_inuse(m, p, leadsize);
       
  4264           leader = chunk2mem(p);
       
  4265         }
       
  4266         p = newp;
       
  4267       }
       
  4268 
       
  4269       /* Give back spare room at the end */
       
  4270       if (!is_mmapped(p)) {
       
  4271         size_t size = chunksize(p);
       
  4272         if (size > nb + MIN_CHUNK_SIZE) {
       
  4273           size_t remainder_size = size - nb;
       
  4274           mchunkptr remainder = chunk_plus_offset(p, nb);
       
  4275           set_inuse(m, p, nb);
       
  4276           set_inuse(m, remainder, remainder_size);
       
  4277           trailer = chunk2mem(remainder);
       
  4278         }
       
  4279       }
       
  4280 
       
  4281       assert (chunksize(p) >= nb);
       
  4282       assert((((size_t)(chunk2mem(p))) % alignment) == 0);
       
  4283       check_inuse_chunk(m, p);
       
  4284       POSTACTION(m);
       
  4285       if (leader != 0) {
       
  4286         internal_free(m, leader);
       
  4287       }
       
  4288       if (trailer != 0) {
       
  4289         internal_free(m, trailer);
       
  4290       }
       
  4291       return chunk2mem(p);
       
  4292     }
       
  4293   }
       
  4294   return 0;
       
  4295 }
       
  4296 
       
  4297 /* ------------------------ comalloc/coalloc support --------------------- */
       
  4298 
       
  4299 static void** ialloc(mstate m,
       
  4300                      size_t n_elements,
       
  4301                      size_t* sizes,
       
  4302                      int opts,
       
  4303                      void* chunks[]) {
       
  4304   /*
       
  4305     This provides common support for independent_X routines, handling
       
  4306     all of the combinations that can result.
       
  4307 
       
  4308     The opts arg has:
       
  4309     bit 0 set if all elements are same size (using sizes[0])
       
  4310     bit 1 set if elements should be zeroed
       
  4311   */
       
  4312 
       
  4313   size_t    element_size;   /* chunksize of each element, if all same */
       
  4314   size_t    contents_size;  /* total size of elements */
       
  4315   size_t    array_size;     /* request size of pointer array */
       
  4316   void*     mem;            /* malloced aggregate space */
       
  4317   mchunkptr p;              /* corresponding chunk */
       
  4318   size_t    remainder_size; /* remaining bytes while splitting */
       
  4319   void**    marray;         /* either "chunks" or malloced ptr array */
       
  4320   mchunkptr array_chunk;    /* chunk for malloced ptr array */
       
  4321   flag_t    was_enabled;    /* to disable mmap */
       
  4322   size_t    size;
       
  4323   size_t    i;
       
  4324 
       
  4325   /* compute array length, if needed */
       
  4326   if (chunks != 0) {
       
  4327     if (n_elements == 0)
       
  4328       return chunks; /* nothing to do */
       
  4329     marray = chunks;
       
  4330     array_size = 0;
       
  4331   }
       
  4332   else {
       
  4333     /* if empty req, must still return chunk representing empty array */
       
  4334     if (n_elements == 0)
       
  4335       return (void**)internal_malloc(m, 0);
       
  4336     marray = 0;
       
  4337     array_size = request2size(n_elements * (sizeof(void*)));
       
  4338   }
       
  4339 
       
  4340   /* compute total element size */
       
  4341   if (opts & 0x1) { /* all-same-size */
       
  4342     element_size = request2size(*sizes);
       
  4343     contents_size = n_elements * element_size;
       
  4344   }
       
  4345   else { /* add up all the sizes */
       
  4346     element_size = 0;
       
  4347     contents_size = 0;
       
  4348     for (i = 0; i != n_elements; ++i)
       
  4349       contents_size += request2size(sizes[i]);
       
  4350   }
       
  4351 
       
  4352   size = contents_size + array_size;
       
  4353 
       
  4354   /*
       
  4355      Allocate the aggregate chunk.  First disable direct-mmapping so
       
  4356      malloc won't use it, since we would not be able to later
       
  4357      free/realloc space internal to a segregated mmap region.
       
  4358   */
       
  4359   was_enabled = use_mmap(m);
       
  4360   disable_mmap(m);
       
  4361   mem = internal_malloc(m, size - CHUNK_OVERHEAD);
       
  4362   if (was_enabled)
       
  4363     enable_mmap(m);
       
  4364   if (mem == 0)
       
  4365     return 0;
       
  4366 
       
  4367   if (PREACTION(m)) return 0;
       
  4368   p = mem2chunk(mem);
       
  4369   remainder_size = chunksize(p);
       
  4370 
       
  4371   assert(!is_mmapped(p));
       
  4372 
       
  4373   if (opts & 0x2) {       /* optionally clear the elements */
       
  4374     memset((size_t*)mem, 0, remainder_size - SIZE_T_SIZE - array_size);
       
  4375   }
       
  4376 
       
  4377   /* If not provided, allocate the pointer array as final part of chunk */
       
  4378   if (marray == 0) {
       
  4379     size_t  array_chunk_size;
       
  4380     array_chunk = chunk_plus_offset(p, contents_size);
       
  4381     array_chunk_size = remainder_size - contents_size;
       
  4382     marray = (void**) (chunk2mem(array_chunk));
       
  4383     set_size_and_pinuse_of_inuse_chunk(m, array_chunk, array_chunk_size);
       
  4384     remainder_size = contents_size;
       
  4385   }
       
  4386 
       
  4387   /* split out elements */
       
  4388   for (i = 0; ; ++i) {
       
  4389     marray[i] = chunk2mem(p);
       
  4390     if (i != n_elements-1) {
       
  4391       if (element_size != 0)
       
  4392         size = element_size;
       
  4393       else
       
  4394         size = request2size(sizes[i]);
       
  4395       remainder_size -= size;
       
  4396       set_size_and_pinuse_of_inuse_chunk(m, p, size);
       
  4397       p = chunk_plus_offset(p, size);
       
  4398     }
       
  4399     else { /* the final element absorbs any overallocation slop */
       
  4400       set_size_and_pinuse_of_inuse_chunk(m, p, remainder_size);
       
  4401       break;
       
  4402     }
       
  4403   }
       
  4404 
       
  4405 #if DEBUG
       
  4406   if (marray != chunks) {
       
  4407     /* final element must have exactly exhausted chunk */
       
  4408     if (element_size != 0) {
       
  4409       assert(remainder_size == element_size);
       
  4410     }
       
  4411     else {
       
  4412       assert(remainder_size == request2size(sizes[i]));
       
  4413     }
       
  4414     check_inuse_chunk(m, mem2chunk(marray));
       
  4415   }
       
  4416   for (i = 0; i != n_elements; ++i)
       
  4417     check_inuse_chunk(m, mem2chunk(marray[i]));
       
  4418 
       
  4419 #endif /* DEBUG */
       
  4420 
       
  4421   POSTACTION(m);
       
  4422   return marray;
       
  4423 }
       
  4424 
       
  4425 
       
  4426 /* -------------------------- public routines ---------------------------- */
       
  4427 
       
  4428 #if !ONLY_MSPACES
       
  4429 
       
  4430 void* dlmalloc(size_t bytes) {
       
  4431   /*
       
  4432      Basic algorithm:
       
  4433      If a small request (< 256 bytes minus per-chunk overhead):
       
  4434        1. If one exists, use a remainderless chunk in associated smallbin.
       
  4435           (Remainderless means that there are too few excess bytes to
       
  4436           represent as a chunk.)
       
  4437        2. If it is big enough, use the dv chunk, which is normally the
       
  4438           chunk adjacent to the one used for the most recent small request.
       
  4439        3. If one exists, split the smallest available chunk in a bin,
       
  4440           saving remainder in dv.
       
  4441        4. If it is big enough, use the top chunk.
       
  4442        5. If available, get memory from system and use it
       
  4443      Otherwise, for a large request:
       
  4444        1. Find the smallest available binned chunk that fits, and use it
       
  4445           if it is better fitting than dv chunk, splitting if necessary.
       
  4446        2. If better fitting than any binned chunk, use the dv chunk.
       
  4447        3. If it is big enough, use the top chunk.
       
  4448        4. If request size >= mmap threshold, try to directly mmap this chunk.
       
  4449        5. If available, get memory from system and use it
       
  4450 
       
  4451      The ugly goto's here ensure that postaction occurs along all paths.
       
  4452   */
       
  4453 
       
  4454   if (!PREACTION(gm)) {
       
  4455     void* mem;
       
  4456     size_t nb;
       
  4457     if (bytes <= MAX_SMALL_REQUEST) {
       
  4458       bindex_t idx;
       
  4459       binmap_t smallbits;
       
  4460       nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes);
       
  4461       idx = small_index(nb);
       
  4462       smallbits = gm->smallmap >> idx;
       
  4463 
       
  4464       if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */
       
  4465         mchunkptr b, p;
       
  4466         idx += ~smallbits & 1;       /* Uses next bin if idx empty */
       
  4467         b = smallbin_at(gm, idx);
       
  4468         p = b->fd;
       
  4469         assert(chunksize(p) == small_index2size(idx));
       
  4470         unlink_first_small_chunk(gm, b, p, idx);
       
  4471         set_inuse_and_pinuse(gm, p, small_index2size(idx));
       
  4472         mem = chunk2mem(p);
       
  4473         check_malloced_chunk(gm, mem, nb);
       
  4474         goto postaction;
       
  4475       }
       
  4476 
       
  4477       else if (nb > gm->dvsize) {
       
  4478         if (smallbits != 0) { /* Use chunk in next nonempty smallbin */
       
  4479           mchunkptr b, p, r;
       
  4480           size_t rsize;
       
  4481           bindex_t i;
       
  4482           binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx));
       
  4483           binmap_t leastbit = least_bit(leftbits);
       
  4484           compute_bit2idx(leastbit, i);
       
  4485           b = smallbin_at(gm, i);
       
  4486           p = b->fd;
       
  4487           assert(chunksize(p) == small_index2size(i));
       
  4488           unlink_first_small_chunk(gm, b, p, i);
       
  4489           rsize = small_index2size(i) - nb;
       
  4490           /* Fit here cannot be remainderless if 4byte sizes */
       
  4491           if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE)
       
  4492             set_inuse_and_pinuse(gm, p, small_index2size(i));
       
  4493           else {
       
  4494             set_size_and_pinuse_of_inuse_chunk(gm, p, nb);
       
  4495             r = chunk_plus_offset(p, nb);
       
  4496             set_size_and_pinuse_of_free_chunk(r, rsize);
       
  4497             replace_dv(gm, r, rsize);
       
  4498           }
       
  4499           mem = chunk2mem(p);
       
  4500           check_malloced_chunk(gm, mem, nb);
       
  4501           goto postaction;
       
  4502         }
       
  4503 
       
  4504         else if (gm->treemap != 0 && (mem = tmalloc_small(gm, nb)) != 0) {
       
  4505           check_malloced_chunk(gm, mem, nb);
       
  4506           goto postaction;
       
  4507         }
       
  4508       }
       
  4509     }
       
  4510     else if (bytes >= MAX_REQUEST)
       
  4511       nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */
       
  4512     else {
       
  4513       nb = pad_request(bytes);
       
  4514       if (gm->treemap != 0 && (mem = tmalloc_large(gm, nb)) != 0) {
       
  4515         check_malloced_chunk(gm, mem, nb);
       
  4516         goto postaction;
       
  4517       }
       
  4518     }
       
  4519 
       
  4520     if (nb <= gm->dvsize) {
       
  4521       size_t rsize = gm->dvsize - nb;
       
  4522       mchunkptr p = gm->dv;
       
  4523       if (rsize >= MIN_CHUNK_SIZE) { /* split dv */
       
  4524         mchunkptr r = gm->dv = chunk_plus_offset(p, nb);
       
  4525         gm->dvsize = rsize;
       
  4526         set_size_and_pinuse_of_free_chunk(r, rsize);
       
  4527         set_size_and_pinuse_of_inuse_chunk(gm, p, nb);
       
  4528       }
       
  4529       else { /* exhaust dv */
       
  4530         size_t dvs = gm->dvsize;
       
  4531         gm->dvsize = 0;
       
  4532         gm->dv = 0;
       
  4533         set_inuse_and_pinuse(gm, p, dvs);
       
  4534       }
       
  4535       mem = chunk2mem(p);
       
  4536       check_malloced_chunk(gm, mem, nb);
       
  4537       goto postaction;
       
  4538     }
       
  4539 
       
  4540     else if (nb < gm->topsize) { /* Split top */
       
  4541       size_t rsize = gm->topsize -= nb;
       
  4542       mchunkptr p = gm->top;
       
  4543       mchunkptr r = gm->top = chunk_plus_offset(p, nb);
       
  4544       r->head = rsize | PINUSE_BIT;
       
  4545       set_size_and_pinuse_of_inuse_chunk(gm, p, nb);
       
  4546       mem = chunk2mem(p);
       
  4547       check_top_chunk(gm, gm->top);
       
  4548       check_malloced_chunk(gm, mem, nb);
       
  4549       goto postaction;
       
  4550     }
       
  4551 
       
  4552     mem = sys_alloc(gm, nb);
       
  4553 
       
  4554   postaction:
       
  4555     POSTACTION(gm);
       
  4556     return mem;
       
  4557   }
       
  4558 
       
  4559   return 0;
       
  4560 }
       
  4561 
       
  4562 void dlfree(void* mem) {
       
  4563   /*
       
  4564      Consolidate freed chunks with preceeding or succeeding bordering
       
  4565      free chunks, if they exist, and then place in a bin.  Intermixed
       
  4566      with special cases for top, dv, mmapped chunks, and usage errors.
       
  4567   */
       
  4568 
       
  4569   if (mem != 0) {
       
  4570     mchunkptr p  = mem2chunk(mem);
       
  4571 #if FOOTERS
       
  4572     mstate fm = get_mstate_for(p);
       
  4573     if (!ok_magic(fm)) {
       
  4574       USAGE_ERROR_ACTION(fm, p);
       
  4575       return;
       
  4576     }
       
  4577 #else /* FOOTERS */
       
  4578 #define fm gm
       
  4579 #endif /* FOOTERS */
       
  4580     if (!PREACTION(fm)) {
       
  4581       check_inuse_chunk(fm, p);
       
  4582       if (RTCHECK(ok_address(fm, p) && ok_cinuse(p))) {
       
  4583         size_t psize = chunksize(p);
       
  4584         mchunkptr next = chunk_plus_offset(p, psize);
       
  4585         if (!pinuse(p)) {
       
  4586           size_t prevsize = p->prev_foot;
       
  4587           if ((prevsize & IS_MMAPPED_BIT) != 0) {
       
  4588             prevsize &= ~IS_MMAPPED_BIT;
       
  4589             psize += prevsize + MMAP_FOOT_PAD;
       
  4590             if (CALL_MUNMAP((char*)p - prevsize, psize) == 0)
       
  4591               fm->footprint -= psize;
       
  4592             goto postaction;
       
  4593           }
       
  4594           else {
       
  4595             mchunkptr prev = chunk_minus_offset(p, prevsize);
       
  4596             psize += prevsize;
       
  4597             p = prev;
       
  4598             if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */
       
  4599               if (p != fm->dv) {
       
  4600                 unlink_chunk(fm, p, prevsize);
       
  4601               }
       
  4602               else if ((next->head & INUSE_BITS) == INUSE_BITS) {
       
  4603                 fm->dvsize = psize;
       
  4604                 set_free_with_pinuse(p, psize, next);
       
  4605                 goto postaction;
       
  4606               }
       
  4607             }
       
  4608             else
       
  4609               goto erroraction;
       
  4610           }
       
  4611         }
       
  4612 
       
  4613         if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) {
       
  4614           if (!cinuse(next)) {  /* consolidate forward */
       
  4615             if (next == fm->top) {
       
  4616               size_t tsize = fm->topsize += psize;
       
  4617               fm->top = p;
       
  4618               p->head = tsize | PINUSE_BIT;
       
  4619               if (p == fm->dv) {
       
  4620                 fm->dv = 0;
       
  4621                 fm->dvsize = 0;
       
  4622               }
       
  4623               if (should_trim(fm, tsize))
       
  4624                 sys_trim(fm, 0);
       
  4625               goto postaction;
       
  4626             }
       
  4627             else if (next == fm->dv) {
       
  4628               size_t dsize = fm->dvsize += psize;
       
  4629               fm->dv = p;
       
  4630               set_size_and_pinuse_of_free_chunk(p, dsize);
       
  4631               goto postaction;
       
  4632             }
       
  4633             else {
       
  4634               size_t nsize = chunksize(next);
       
  4635               psize += nsize;
       
  4636               unlink_chunk(fm, next, nsize);
       
  4637               set_size_and_pinuse_of_free_chunk(p, psize);
       
  4638               if (p == fm->dv) {
       
  4639                 fm->dvsize = psize;
       
  4640                 goto postaction;
       
  4641               }
       
  4642             }
       
  4643           }
       
  4644           else
       
  4645             set_free_with_pinuse(p, psize, next);
       
  4646 
       
  4647           if (is_small(psize)) {
       
  4648             insert_small_chunk(fm, p, psize);
       
  4649             check_free_chunk(fm, p);
       
  4650           }
       
  4651           else {
       
  4652             tchunkptr tp = (tchunkptr)p;
       
  4653             insert_large_chunk(fm, tp, psize);
       
  4654             check_free_chunk(fm, p);
       
  4655             if (--fm->release_checks == 0)
       
  4656               release_unused_segments(fm);
       
  4657           }
       
  4658           goto postaction;
       
  4659         }
       
  4660       }
       
  4661     erroraction:
       
  4662       USAGE_ERROR_ACTION(fm, p);
       
  4663     postaction:
       
  4664       POSTACTION(fm);
       
  4665     }
       
  4666   }
       
  4667 #if !FOOTERS
       
  4668 #undef fm
       
  4669 #endif /* FOOTERS */
       
  4670 }
       
  4671 
       
  4672 void* dlcalloc(size_t n_elements, size_t elem_size) {
       
  4673   void* mem;
       
  4674   size_t req = 0;
       
  4675   if (n_elements != 0) {
       
  4676     req = n_elements * elem_size;
       
  4677     if (((n_elements | elem_size) & ~(size_t)0xffff) &&
       
  4678         (req / n_elements != elem_size))
       
  4679       req = MAX_SIZE_T; /* force downstream failure on overflow */
       
  4680   }
       
  4681   mem = dlmalloc(req);
       
  4682   if (mem != 0 && calloc_must_clear(mem2chunk(mem)))
       
  4683     memset(mem, 0, req);
       
  4684   return mem;
       
  4685 }
       
  4686 
       
  4687 void* dlrealloc(void* oldmem, size_t bytes) {
       
  4688   if (oldmem == 0)
       
  4689     return dlmalloc(bytes);
       
  4690 #ifdef REALLOC_ZERO_BYTES_FREES
       
  4691   if (bytes == 0) {
       
  4692     dlfree(oldmem);
       
  4693     return 0;
       
  4694   }
       
  4695 #endif /* REALLOC_ZERO_BYTES_FREES */
       
  4696   else {
       
  4697 #if ! FOOTERS
       
  4698     mstate m = gm;
       
  4699 #else /* FOOTERS */
       
  4700     mstate m = get_mstate_for(mem2chunk(oldmem));
       
  4701     if (!ok_magic(m)) {
       
  4702       USAGE_ERROR_ACTION(m, oldmem);
       
  4703       return 0;
       
  4704     }
       
  4705 #endif /* FOOTERS */
       
  4706     return internal_realloc(m, oldmem, bytes);
       
  4707   }
       
  4708 }
       
  4709 
       
  4710 void* dlmemalign(size_t alignment, size_t bytes) {
       
  4711   return internal_memalign(gm, alignment, bytes);
       
  4712 }
       
  4713 
       
  4714 void** dlindependent_calloc(size_t n_elements, size_t elem_size,
       
  4715                                  void* chunks[]) {
       
  4716   size_t sz = elem_size; /* serves as 1-element array */
       
  4717   return ialloc(gm, n_elements, &sz, 3, chunks);
       
  4718 }
       
  4719 
       
  4720 void** dlindependent_comalloc(size_t n_elements, size_t sizes[],
       
  4721                                    void* chunks[]) {
       
  4722   return ialloc(gm, n_elements, sizes, 0, chunks);
       
  4723 }
       
  4724 
       
  4725 void* dlvalloc(size_t bytes) {
       
  4726   size_t pagesz;
       
  4727   init_mparams();
       
  4728   pagesz = mparams.page_size;
       
  4729   return dlmemalign(pagesz, bytes);
       
  4730 }
       
  4731 
       
  4732 void* dlpvalloc(size_t bytes) {
       
  4733   size_t pagesz;
       
  4734   init_mparams();
       
  4735   pagesz = mparams.page_size;
       
  4736   return dlmemalign(pagesz, (bytes + pagesz - SIZE_T_ONE) & ~(pagesz - SIZE_T_ONE));
       
  4737 }
       
  4738 
       
  4739 int dlmalloc_trim(size_t pad) {
       
  4740   int result = 0;
       
  4741   if (!PREACTION(gm)) {
       
  4742     result = sys_trim(gm, pad);
       
  4743     POSTACTION(gm);
       
  4744   }
       
  4745   return result;
       
  4746 }
       
  4747 
       
  4748 size_t dlmalloc_footprint(void) {
       
  4749   return gm->footprint;
       
  4750 }
       
  4751 
       
  4752 size_t dlmalloc_max_footprint(void) {
       
  4753   return gm->max_footprint;
       
  4754 }
       
  4755 
       
  4756 #if !NO_MALLINFO
       
  4757 struct mallinfo dlmallinfo(void) {
       
  4758   return internal_mallinfo(gm);
       
  4759 }
       
  4760 #endif /* NO_MALLINFO */
       
  4761 
       
  4762 void dlmalloc_stats() {
       
  4763   internal_malloc_stats(gm);
       
  4764 }
       
  4765 
       
  4766 size_t dlmalloc_usable_size(void* mem) {
       
  4767   if (mem != 0) {
       
  4768     mchunkptr p = mem2chunk(mem);
       
  4769     if (cinuse(p))
       
  4770       return chunksize(p) - overhead_for(p);
       
  4771   }
       
  4772   return 0;
       
  4773 }
       
  4774 
       
  4775 int dlmallopt(int param_number, int value) {
       
  4776   return change_mparam(param_number, value);
       
  4777 }
       
  4778 
       
  4779 #endif /* !ONLY_MSPACES */
       
  4780 
       
  4781 /* ----------------------------- user mspaces ---------------------------- */
       
  4782 
       
  4783 #if MSPACES
       
  4784 
       
  4785 static mstate init_user_mstate(char* tbase, size_t tsize) {
       
  4786   size_t msize = pad_request(sizeof(struct malloc_state));
       
  4787   mchunkptr mn;
       
  4788   mchunkptr msp = align_as_chunk(tbase);
       
  4789   mstate m = (mstate)(chunk2mem(msp));
       
  4790   memset(m, 0, msize);
       
  4791   INITIAL_LOCK(&m->mutex);
       
  4792   msp->head = (msize|PINUSE_BIT|CINUSE_BIT);
       
  4793   m->seg.base = m->least_addr = tbase;
       
  4794   m->seg.size = m->footprint = m->max_footprint = tsize;
       
  4795   m->magic = mparams.magic;
       
  4796   m->release_checks = MAX_RELEASE_CHECK_RATE;
       
  4797   m->mflags = mparams.default_mflags;
       
  4798   m->extp = 0;
       
  4799   m->exts = 0;
       
  4800   disable_contiguous(m);
       
  4801   init_bins(m);
       
  4802   mn = next_chunk(mem2chunk(m));
       
  4803   init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) - TOP_FOOT_SIZE);
       
  4804   check_top_chunk(m, m->top);
       
  4805   return m;
       
  4806 }
       
  4807 
       
  4808 mspace create_mspace(size_t capacity, int locked) {
       
  4809   mstate m = 0;
       
  4810   size_t msize = pad_request(sizeof(struct malloc_state));
       
  4811   init_mparams(); /* Ensure pagesize etc initialized */
       
  4812 
       
  4813   if (capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) {
       
  4814     size_t rs = ((capacity == 0)? mparams.granularity :
       
  4815                  (capacity + TOP_FOOT_SIZE + msize));
       
  4816     size_t tsize = granularity_align(rs);
       
  4817     char* tbase = (char*)(CALL_MMAP(tsize));
       
  4818     if (tbase != CMFAIL) {
       
  4819       m = init_user_mstate(tbase, tsize);
       
  4820       m->seg.sflags = IS_MMAPPED_BIT;
       
  4821       set_lock(m, locked);
       
  4822     }
       
  4823   }
       
  4824   return (mspace)m;
       
  4825 }
       
  4826 
       
  4827 mspace create_mspace_with_base(void* base, size_t capacity, int locked) {
       
  4828   mstate m = 0;
       
  4829   size_t msize = pad_request(sizeof(struct malloc_state));
       
  4830   init_mparams(); /* Ensure pagesize etc initialized */
       
  4831 
       
  4832   if (capacity > msize + TOP_FOOT_SIZE &&
       
  4833       capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) {
       
  4834     m = init_user_mstate((char*)base, capacity);
       
  4835     m->seg.sflags = EXTERN_BIT;
       
  4836     set_lock(m, locked);
       
  4837   }
       
  4838   return (mspace)m;
       
  4839 }
       
  4840 
       
  4841 size_t destroy_mspace(mspace msp) {
       
  4842   size_t freed = 0;
       
  4843   mstate ms = (mstate)msp;
       
  4844   if (ok_magic(ms)) {
       
  4845     msegmentptr sp = &ms->seg;
       
  4846     while (sp != 0) {
       
  4847       char* base = sp->base;
       
  4848       size_t size = sp->size;
       
  4849       flag_t flag = sp->sflags;
       
  4850       sp = sp->next;
       
  4851       if ((flag & IS_MMAPPED_BIT) && !(flag & EXTERN_BIT) &&
       
  4852           CALL_MUNMAP(base, size) == 0)
       
  4853         freed += size;
       
  4854     }
       
  4855   }
       
  4856   else {
       
  4857     USAGE_ERROR_ACTION(ms,ms);
       
  4858   }
       
  4859   return freed;
       
  4860 }
       
  4861 
       
  4862 /*
       
  4863   mspace versions of routines are near-clones of the global
       
  4864   versions. This is not so nice but better than the alternatives.
       
  4865 */
       
  4866 
       
  4867 
       
  4868 void* mspace_malloc(mspace msp, size_t bytes) {
       
  4869   mstate ms = (mstate)msp;
       
  4870   if (!ok_magic(ms)) {
       
  4871     USAGE_ERROR_ACTION(ms,ms);
       
  4872     return 0;
       
  4873   }
       
  4874   if (!PREACTION(ms)) {
       
  4875     void* mem;
       
  4876     size_t nb;
       
  4877     if (bytes <= MAX_SMALL_REQUEST) {
       
  4878       bindex_t idx;
       
  4879       binmap_t smallbits;
       
  4880       nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes);
       
  4881       idx = small_index(nb);
       
  4882       smallbits = ms->smallmap >> idx;
       
  4883 
       
  4884       if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */
       
  4885         mchunkptr b, p;
       
  4886         idx += ~smallbits & 1;       /* Uses next bin if idx empty */
       
  4887         b = smallbin_at(ms, idx);
       
  4888         p = b->fd;
       
  4889         assert(chunksize(p) == small_index2size(idx));
       
  4890         unlink_first_small_chunk(ms, b, p, idx);
       
  4891         set_inuse_and_pinuse(ms, p, small_index2size(idx));
       
  4892         mem = chunk2mem(p);
       
  4893         check_malloced_chunk(ms, mem, nb);
       
  4894         goto postaction;
       
  4895       }
       
  4896 
       
  4897       else if (nb > ms->dvsize) {
       
  4898         if (smallbits != 0) { /* Use chunk in next nonempty smallbin */
       
  4899           mchunkptr b, p, r;
       
  4900           size_t rsize;
       
  4901           bindex_t i;
       
  4902           binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx));
       
  4903           binmap_t leastbit = least_bit(leftbits);
       
  4904           compute_bit2idx(leastbit, i);
       
  4905           b = smallbin_at(ms, i);
       
  4906           p = b->fd;
       
  4907           assert(chunksize(p) == small_index2size(i));
       
  4908           unlink_first_small_chunk(ms, b, p, i);
       
  4909           rsize = small_index2size(i) - nb;
       
  4910           /* Fit here cannot be remainderless if 4byte sizes */
       
  4911           if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE)
       
  4912             set_inuse_and_pinuse(ms, p, small_index2size(i));
       
  4913           else {
       
  4914             set_size_and_pinuse_of_inuse_chunk(ms, p, nb);
       
  4915             r = chunk_plus_offset(p, nb);
       
  4916             set_size_and_pinuse_of_free_chunk(r, rsize);
       
  4917             replace_dv(ms, r, rsize);
       
  4918           }
       
  4919           mem = chunk2mem(p);
       
  4920           check_malloced_chunk(ms, mem, nb);
       
  4921           goto postaction;
       
  4922         }
       
  4923 
       
  4924         else if (ms->treemap != 0 && (mem = tmalloc_small(ms, nb)) != 0) {
       
  4925           check_malloced_chunk(ms, mem, nb);
       
  4926           goto postaction;
       
  4927         }
       
  4928       }
       
  4929     }
       
  4930     else if (bytes >= MAX_REQUEST)
       
  4931       nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */
       
  4932     else {
       
  4933       nb = pad_request(bytes);
       
  4934       if (ms->treemap != 0 && (mem = tmalloc_large(ms, nb)) != 0) {
       
  4935         check_malloced_chunk(ms, mem, nb);
       
  4936         goto postaction;
       
  4937       }
       
  4938     }
       
  4939 
       
  4940     if (nb <= ms->dvsize) {
       
  4941       size_t rsize = ms->dvsize - nb;
       
  4942       mchunkptr p = ms->dv;
       
  4943       if (rsize >= MIN_CHUNK_SIZE) { /* split dv */
       
  4944         mchunkptr r = ms->dv = chunk_plus_offset(p, nb);
       
  4945         ms->dvsize = rsize;
       
  4946         set_size_and_pinuse_of_free_chunk(r, rsize);
       
  4947         set_size_and_pinuse_of_inuse_chunk(ms, p, nb);
       
  4948       }
       
  4949       else { /* exhaust dv */
       
  4950         size_t dvs = ms->dvsize;
       
  4951         ms->dvsize = 0;
       
  4952         ms->dv = 0;
       
  4953         set_inuse_and_pinuse(ms, p, dvs);
       
  4954       }
       
  4955       mem = chunk2mem(p);
       
  4956       check_malloced_chunk(ms, mem, nb);
       
  4957       goto postaction;
       
  4958     }
       
  4959 
       
  4960     else if (nb < ms->topsize) { /* Split top */
       
  4961       size_t rsize = ms->topsize -= nb;
       
  4962       mchunkptr p = ms->top;
       
  4963       mchunkptr r = ms->top = chunk_plus_offset(p, nb);
       
  4964       r->head = rsize | PINUSE_BIT;
       
  4965       set_size_and_pinuse_of_inuse_chunk(ms, p, nb);
       
  4966       mem = chunk2mem(p);
       
  4967       check_top_chunk(ms, ms->top);
       
  4968       check_malloced_chunk(ms, mem, nb);
       
  4969       goto postaction;
       
  4970     }
       
  4971 
       
  4972     mem = sys_alloc(ms, nb);
       
  4973 
       
  4974   postaction:
       
  4975     POSTACTION(ms);
       
  4976     return mem;
       
  4977   }
       
  4978 
       
  4979   return 0;
       
  4980 }
       
  4981 
       
  4982 void mspace_free(mspace msp, void* mem) {
       
  4983   if (mem != 0) {
       
  4984     mchunkptr p  = mem2chunk(mem);
       
  4985 #if FOOTERS
       
  4986     mstate fm = get_mstate_for(p);
       
  4987 #else /* FOOTERS */
       
  4988     mstate fm = (mstate)msp;
       
  4989 #endif /* FOOTERS */
       
  4990     if (!ok_magic(fm)) {
       
  4991       USAGE_ERROR_ACTION(fm, p);
       
  4992       return;
       
  4993     }
       
  4994     if (!PREACTION(fm)) {
       
  4995       check_inuse_chunk(fm, p);
       
  4996       if (RTCHECK(ok_address(fm, p) && ok_cinuse(p))) {
       
  4997         size_t psize = chunksize(p);
       
  4998         mchunkptr next = chunk_plus_offset(p, psize);
       
  4999         if (!pinuse(p)) {
       
  5000           size_t prevsize = p->prev_foot;
       
  5001           if ((prevsize & IS_MMAPPED_BIT) != 0) {
       
  5002             prevsize &= ~IS_MMAPPED_BIT;
       
  5003             psize += prevsize + MMAP_FOOT_PAD;
       
  5004             if (CALL_MUNMAP((char*)p - prevsize, psize) == 0)
       
  5005               fm->footprint -= psize;
       
  5006             goto postaction;
       
  5007           }
       
  5008           else {
       
  5009             mchunkptr prev = chunk_minus_offset(p, prevsize);
       
  5010             psize += prevsize;
       
  5011             p = prev;
       
  5012             if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */
       
  5013               if (p != fm->dv) {
       
  5014                 unlink_chunk(fm, p, prevsize);
       
  5015               }
       
  5016               else if ((next->head & INUSE_BITS) == INUSE_BITS) {
       
  5017                 fm->dvsize = psize;
       
  5018                 set_free_with_pinuse(p, psize, next);
       
  5019                 goto postaction;
       
  5020               }
       
  5021             }
       
  5022             else
       
  5023               goto erroraction;
       
  5024           }
       
  5025         }
       
  5026 
       
  5027         if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) {
       
  5028           if (!cinuse(next)) {  /* consolidate forward */
       
  5029             if (next == fm->top) {
       
  5030               size_t tsize = fm->topsize += psize;
       
  5031               fm->top = p;
       
  5032               p->head = tsize | PINUSE_BIT;
       
  5033               if (p == fm->dv) {
       
  5034                 fm->dv = 0;
       
  5035                 fm->dvsize = 0;
       
  5036               }
       
  5037               if (should_trim(fm, tsize))
       
  5038                 sys_trim(fm, 0);
       
  5039               goto postaction;
       
  5040             }
       
  5041             else if (next == fm->dv) {
       
  5042               size_t dsize = fm->dvsize += psize;
       
  5043               fm->dv = p;
       
  5044               set_size_and_pinuse_of_free_chunk(p, dsize);
       
  5045               goto postaction;
       
  5046             }
       
  5047             else {
       
  5048               size_t nsize = chunksize(next);
       
  5049               psize += nsize;
       
  5050               unlink_chunk(fm, next, nsize);
       
  5051               set_size_and_pinuse_of_free_chunk(p, psize);
       
  5052               if (p == fm->dv) {
       
  5053                 fm->dvsize = psize;
       
  5054                 goto postaction;
       
  5055               }
       
  5056             }
       
  5057           }
       
  5058           else
       
  5059             set_free_with_pinuse(p, psize, next);
       
  5060 
       
  5061           if (is_small(psize)) {
       
  5062             insert_small_chunk(fm, p, psize);
       
  5063             check_free_chunk(fm, p);
       
  5064           }
       
  5065           else {
       
  5066             tchunkptr tp = (tchunkptr)p;
       
  5067             insert_large_chunk(fm, tp, psize);
       
  5068             check_free_chunk(fm, p);
       
  5069             if (--fm->release_checks == 0)
       
  5070               release_unused_segments(fm);
       
  5071           }
       
  5072           goto postaction;
       
  5073         }
       
  5074       }
       
  5075     erroraction:
       
  5076       USAGE_ERROR_ACTION(fm, p);
       
  5077     postaction:
       
  5078       POSTACTION(fm);
       
  5079     }
       
  5080   }
       
  5081 }
       
  5082 
       
  5083 void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size) {
       
  5084   void* mem;
       
  5085   size_t req = 0;
       
  5086   mstate ms = (mstate)msp;
       
  5087   if (!ok_magic(ms)) {
       
  5088     USAGE_ERROR_ACTION(ms,ms);
       
  5089     return 0;
       
  5090   }
       
  5091   if (n_elements != 0) {
       
  5092     req = n_elements * elem_size;
       
  5093     if (((n_elements | elem_size) & ~(size_t)0xffff) &&
       
  5094         (req / n_elements != elem_size))
       
  5095       req = MAX_SIZE_T; /* force downstream failure on overflow */
       
  5096   }
       
  5097   mem = internal_malloc(ms, req);
       
  5098   if (mem != 0 && calloc_must_clear(mem2chunk(mem)))
       
  5099     memset(mem, 0, req);
       
  5100   return mem;
       
  5101 }
       
  5102 
       
  5103 void* mspace_realloc(mspace msp, void* oldmem, size_t bytes) {
       
  5104   if (oldmem == 0)
       
  5105     return mspace_malloc(msp, bytes);
       
  5106 #ifdef REALLOC_ZERO_BYTES_FREES
       
  5107   if (bytes == 0) {
       
  5108     mspace_free(msp, oldmem);
       
  5109     return 0;
       
  5110   }
       
  5111 #endif /* REALLOC_ZERO_BYTES_FREES */
       
  5112   else {
       
  5113 #if FOOTERS
       
  5114     mchunkptr p  = mem2chunk(oldmem);
       
  5115     mstate ms = get_mstate_for(p);
       
  5116 #else /* FOOTERS */
       
  5117     mstate ms = (mstate)msp;
       
  5118 #endif /* FOOTERS */
       
  5119     if (!ok_magic(ms)) {
       
  5120       USAGE_ERROR_ACTION(ms,ms);
       
  5121       return 0;
       
  5122     }
       
  5123     return internal_realloc(ms, oldmem, bytes);
       
  5124   }
       
  5125 }
       
  5126 
       
  5127 void* mspace_memalign(mspace msp, size_t alignment, size_t bytes) {
       
  5128   mstate ms = (mstate)msp;
       
  5129   if (!ok_magic(ms)) {
       
  5130     USAGE_ERROR_ACTION(ms,ms);
       
  5131     return 0;
       
  5132   }
       
  5133   return internal_memalign(ms, alignment, bytes);
       
  5134 }
       
  5135 
       
  5136 void** mspace_independent_calloc(mspace msp, size_t n_elements,
       
  5137                                  size_t elem_size, void* chunks[]) {
       
  5138   size_t sz = elem_size; /* serves as 1-element array */
       
  5139   mstate ms = (mstate)msp;
       
  5140   if (!ok_magic(ms)) {
       
  5141     USAGE_ERROR_ACTION(ms,ms);
       
  5142     return 0;
       
  5143   }
       
  5144   return ialloc(ms, n_elements, &sz, 3, chunks);
       
  5145 }
       
  5146 
       
  5147 void** mspace_independent_comalloc(mspace msp, size_t n_elements,
       
  5148                                    size_t sizes[], void* chunks[]) {
       
  5149   mstate ms = (mstate)msp;
       
  5150   if (!ok_magic(ms)) {
       
  5151     USAGE_ERROR_ACTION(ms,ms);
       
  5152     return 0;
       
  5153   }
       
  5154   return ialloc(ms, n_elements, sizes, 0, chunks);
       
  5155 }
       
  5156 
       
  5157 int mspace_trim(mspace msp, size_t pad) {
       
  5158   int result = 0;
       
  5159   mstate ms = (mstate)msp;
       
  5160   if (ok_magic(ms)) {
       
  5161     if (!PREACTION(ms)) {
       
  5162       result = sys_trim(ms, pad);
       
  5163       POSTACTION(ms);
       
  5164     }
       
  5165   }
       
  5166   else {
       
  5167     USAGE_ERROR_ACTION(ms,ms);
       
  5168   }
       
  5169   return result;
       
  5170 }
       
  5171 
       
  5172 void mspace_malloc_stats(mspace msp) {
       
  5173   mstate ms = (mstate)msp;
       
  5174   if (ok_magic(ms)) {
       
  5175     internal_malloc_stats(ms);
       
  5176   }
       
  5177   else {
       
  5178     USAGE_ERROR_ACTION(ms,ms);
       
  5179   }
       
  5180 }
       
  5181 
       
  5182 size_t mspace_footprint(mspace msp) {
       
  5183   size_t result = 0;
       
  5184   mstate ms = (mstate)msp;
       
  5185   if (ok_magic(ms)) {
       
  5186     result = ms->footprint;
       
  5187   }
       
  5188   else {
       
  5189     USAGE_ERROR_ACTION(ms,ms);
       
  5190   }
       
  5191   return result;
       
  5192 }
       
  5193 
       
  5194 
       
  5195 size_t mspace_max_footprint(mspace msp) {
       
  5196   size_t result = 0;
       
  5197   mstate ms = (mstate)msp;
       
  5198   if (ok_magic(ms)) {
       
  5199     result = ms->max_footprint;
       
  5200   }
       
  5201   else {
       
  5202     USAGE_ERROR_ACTION(ms,ms);
       
  5203   }
       
  5204   return result;
       
  5205 }
       
  5206 
       
  5207 
       
  5208 #if !NO_MALLINFO
       
  5209 struct mallinfo mspace_mallinfo(mspace msp) {
       
  5210   mstate ms = (mstate)msp;
       
  5211   if (!ok_magic(ms)) {
       
  5212     USAGE_ERROR_ACTION(ms,ms);
       
  5213   }
       
  5214   return internal_mallinfo(ms);
       
  5215 }
       
  5216 #endif /* NO_MALLINFO */
       
  5217 
       
  5218 size_t mspace_usable_size(void* mem) {
       
  5219   if (mem != 0) {
       
  5220     mchunkptr p = mem2chunk(mem);
       
  5221     if (cinuse(p))
       
  5222       return chunksize(p) - overhead_for(p);
       
  5223   }
       
  5224   return 0;
       
  5225 }
       
  5226 
       
  5227 int mspace_mallopt(int param_number, int value) {
       
  5228   return change_mparam(param_number, value);
       
  5229 }
       
  5230 
       
  5231 #endif /* MSPACES */
       
  5232 
       
  5233 /* -------------------- Alternative MORECORE functions ------------------- */
       
  5234 
       
  5235 /*
       
  5236   Guidelines for creating a custom version of MORECORE:
       
  5237 
       
  5238   * For best performance, MORECORE should allocate in multiples of pagesize.
       
  5239   * MORECORE may allocate more memory than requested. (Or even less,
       
  5240       but this will usually result in a malloc failure.)
       
  5241   * MORECORE must not allocate memory when given argument zero, but
       
  5242       instead return one past the end address of memory from previous
       
  5243       nonzero call.
       
  5244   * For best performance, consecutive calls to MORECORE with positive
       
  5245       arguments should return increasing addresses, indicating that
       
  5246       space has been contiguously extended.
       
  5247   * Even though consecutive calls to MORECORE need not return contiguous
       
  5248       addresses, it must be OK for malloc'ed chunks to span multiple
       
  5249       regions in those cases where they do happen to be contiguous.
       
  5250   * MORECORE need not handle negative arguments -- it may instead
       
  5251       just return MFAIL when given negative arguments.
       
  5252       Negative arguments are always multiples of pagesize. MORECORE
       
  5253       must not misinterpret negative args as large positive unsigned
       
  5254       args. You can suppress all such calls from even occurring by defining
       
  5255       MORECORE_CANNOT_TRIM,
       
  5256 
       
  5257   As an example alternative MORECORE, here is a custom allocator
       
  5258   kindly contributed for pre-OSX macOS.  It uses virtually but not
       
  5259   necessarily physically contiguous non-paged memory (locked in,
       
  5260   present and won't get swapped out).  You can use it by uncommenting
       
  5261   this section, adding some #includes, and setting up the appropriate
       
  5262   defines above:
       
  5263 
       
  5264       #define MORECORE osMoreCore
       
  5265 
       
  5266   There is also a shutdown routine that should somehow be called for
       
  5267   cleanup upon program exit.
       
  5268 
       
  5269   #define MAX_POOL_ENTRIES 100
       
  5270   #define MINIMUM_MORECORE_SIZE  (64 * 1024U)
       
  5271   static int next_os_pool;
       
  5272   void *our_os_pools[MAX_POOL_ENTRIES];
       
  5273 
       
  5274   void *osMoreCore(int size)
       
  5275   {
       
  5276     void *ptr = 0;
       
  5277     static void *sbrk_top = 0;
       
  5278 
       
  5279     if (size > 0)
       
  5280     {
       
  5281       if (size < MINIMUM_MORECORE_SIZE)
       
  5282          size = MINIMUM_MORECORE_SIZE;
       
  5283       if (CurrentExecutionLevel() == kTaskLevel)
       
  5284          ptr = PoolAllocateResident(size + RM_PAGE_SIZE, 0);
       
  5285       if (ptr == 0)
       
  5286       {
       
  5287         return (void *) MFAIL;
       
  5288       }
       
  5289       // save ptrs so they can be freed during cleanup
       
  5290       our_os_pools[next_os_pool] = ptr;
       
  5291       next_os_pool++;
       
  5292       ptr = (void *) ((((size_t) ptr) + RM_PAGE_MASK) & ~RM_PAGE_MASK);
       
  5293       sbrk_top = (char *) ptr + size;
       
  5294       return ptr;
       
  5295     }
       
  5296     else if (size < 0)
       
  5297     {
       
  5298       // we don't currently support shrink behavior
       
  5299       return (void *) MFAIL;
       
  5300     }
       
  5301     else
       
  5302     {
       
  5303       return sbrk_top;
       
  5304     }
       
  5305   }
       
  5306 
       
  5307   // cleanup any allocated memory pools
       
  5308   // called as last thing before shutting down driver
       
  5309 
       
  5310   void osCleanupMem(void)
       
  5311   {
       
  5312     void **ptr;
       
  5313 
       
  5314     for (ptr = our_os_pools; ptr < &our_os_pools[MAX_POOL_ENTRIES]; ptr++)
       
  5315       if (*ptr)
       
  5316       {
       
  5317          PoolDeallocate(*ptr);
       
  5318          *ptr = 0;
       
  5319       }
       
  5320   }
       
  5321 
       
  5322 */
       
  5323 
       
  5324 
       
  5325 /* -----------------------------------------------------------------------
       
  5326 History:
       
  5327     V2.8.4 (not yet released)
       
  5328       * Fix bad error check in mspace_footprint
       
  5329       * Adaptations for ptmalloc, courtesy of Wolfram Gloger.
       
  5330       * Reentrant spin locks, courtesy of Earl Chew and others
       
  5331       * Win32 improvements, courtesy of Niall Douglas and Earl Chew
       
  5332       * Add NO_SEGMENT_TRAVERSAL and MAX_RELEASE_CHECK_RATE options
       
  5333       * Various small adjustments to reduce warnings on some compilers
       
  5334       * Extension hook in malloc_state
       
  5335 
       
  5336     V2.8.3 Thu Sep 22 11:16:32 2005  Doug Lea  (dl at gee)
       
  5337       * Add max_footprint functions
       
  5338       * Ensure all appropriate literals are size_t
       
  5339       * Fix conditional compilation problem for some #define settings
       
  5340       * Avoid concatenating segments with the one provided
       
  5341         in create_mspace_with_base
       
  5342       * Rename some variables to avoid compiler shadowing warnings
       
  5343       * Use explicit lock initialization.
       
  5344       * Better handling of sbrk interference.
       
  5345       * Simplify and fix segment insertion, trimming and mspace_destroy
       
  5346       * Reinstate REALLOC_ZERO_BYTES_FREES option from 2.7.x
       
  5347       * Thanks especially to Dennis Flanagan for help on these.
       
  5348 
       
  5349     V2.8.2 Sun Jun 12 16:01:10 2005  Doug Lea  (dl at gee)
       
  5350       * Fix memalign brace error.
       
  5351 
       
  5352     V2.8.1 Wed Jun  8 16:11:46 2005  Doug Lea  (dl at gee)
       
  5353       * Fix improper #endif nesting in C++
       
  5354       * Add explicit casts needed for C++
       
  5355 
       
  5356     V2.8.0 Mon May 30 14:09:02 2005  Doug Lea  (dl at gee)
       
  5357       * Use trees for large bins
       
  5358       * Support mspaces
       
  5359       * Use segments to unify sbrk-based and mmap-based system allocation,
       
  5360         removing need for emulation on most platforms without sbrk.
       
  5361       * Default safety checks
       
  5362       * Optional footer checks. Thanks to William Robertson for the idea.
       
  5363       * Internal code refactoring
       
  5364       * Incorporate suggestions and platform-specific changes.
       
  5365         Thanks to Dennis Flanagan, Colin Plumb, Niall Douglas,
       
  5366         Aaron Bachmann,  Emery Berger, and others.
       
  5367       * Speed up non-fastbin processing enough to remove fastbins.
       
  5368       * Remove useless cfree() to avoid conflicts with other apps.
       
  5369       * Remove internal memcpy, memset. Compilers handle builtins better.
       
  5370       * Remove some options that no one ever used and rename others.
       
  5371 
       
  5372     V2.7.2 Sat Aug 17 09:07:30 2002  Doug Lea  (dl at gee)
       
  5373       * Fix malloc_state bitmap array misdeclaration
       
  5374 
       
  5375     V2.7.1 Thu Jul 25 10:58:03 2002  Doug Lea  (dl at gee)
       
  5376       * Allow tuning of FIRST_SORTED_BIN_SIZE
       
  5377       * Use PTR_UINT as type for all ptr->int casts. Thanks to John Belmonte.
       
  5378       * Better detection and support for non-contiguousness of MORECORE.
       
  5379         Thanks to Andreas Mueller, Conal Walsh, and Wolfram Gloger
       
  5380       * Bypass most of malloc if no frees. Thanks To Emery Berger.
       
  5381       * Fix freeing of old top non-contiguous chunk im sysmalloc.
       
  5382       * Raised default trim and map thresholds to 256K.
       
  5383       * Fix mmap-related #defines. Thanks to Lubos Lunak.
       
  5384       * Fix copy macros; added LACKS_FCNTL_H. Thanks to Neal Walfield.
       
  5385       * Branch-free bin calculation
       
  5386       * Default trim and mmap thresholds now 256K.
       
  5387 
       
  5388     V2.7.0 Sun Mar 11 14:14:06 2001  Doug Lea  (dl at gee)
       
  5389       * Introduce independent_comalloc and independent_calloc.
       
  5390         Thanks to Michael Pachos for motivation and help.
       
  5391       * Make optional .h file available
       
  5392       * Allow > 2GB requests on 32bit systems.
       
  5393       * new WIN32 sbrk, mmap, munmap, lock code from <Walter@GeNeSys-e.de>.
       
  5394         Thanks also to Andreas Mueller <a.mueller at paradatec.de>,
       
  5395         and Anonymous.
       
  5396       * Allow override of MALLOC_ALIGNMENT (Thanks to Ruud Waij for
       
  5397         helping test this.)
       
  5398       * memalign: check alignment arg
       
  5399       * realloc: don't try to shift chunks backwards, since this
       
  5400         leads to  more fragmentation in some programs and doesn't
       
  5401         seem to help in any others.
       
  5402       * Collect all cases in malloc requiring system memory into sysmalloc
       
  5403       * Use mmap as backup to sbrk
       
  5404       * Place all internal state in malloc_state
       
  5405       * Introduce fastbins (although similar to 2.5.1)
       
  5406       * Many minor tunings and cosmetic improvements
       
  5407       * Introduce USE_PUBLIC_MALLOC_WRAPPERS, USE_MALLOC_LOCK
       
  5408       * Introduce MALLOC_FAILURE_ACTION, MORECORE_CONTIGUOUS
       
  5409         Thanks to Tony E. Bennett <tbennett@nvidia.com> and others.
       
  5410       * Include errno.h to support default failure action.
       
  5411 
       
  5412     V2.6.6 Sun Dec  5 07:42:19 1999  Doug Lea  (dl at gee)
       
  5413       * return null for negative arguments
       
  5414       * Added Several WIN32 cleanups from Martin C. Fong <mcfong at yahoo.com>
       
  5415          * Add 'LACKS_SYS_PARAM_H' for those systems without 'sys/param.h'
       
  5416           (e.g. WIN32 platforms)
       
  5417          * Cleanup header file inclusion for WIN32 platforms
       
  5418          * Cleanup code to avoid Microsoft Visual C++ compiler complaints
       
  5419          * Add 'USE_DL_PREFIX' to quickly allow co-existence with existing
       
  5420            memory allocation routines
       
  5421          * Set 'malloc_getpagesize' for WIN32 platforms (needs more work)
       
  5422          * Use 'assert' rather than 'ASSERT' in WIN32 code to conform to
       
  5423            usage of 'assert' in non-WIN32 code
       
  5424          * Improve WIN32 'sbrk()' emulation's 'findRegion()' routine to
       
  5425            avoid infinite loop
       
  5426       * Always call 'fREe()' rather than 'free()'
       
  5427 
       
  5428     V2.6.5 Wed Jun 17 15:57:31 1998  Doug Lea  (dl at gee)
       
  5429       * Fixed ordering problem with boundary-stamping
       
  5430 
       
  5431     V2.6.3 Sun May 19 08:17:58 1996  Doug Lea  (dl at gee)
       
  5432       * Added pvalloc, as recommended by H.J. Liu
       
  5433       * Added 64bit pointer support mainly from Wolfram Gloger
       
  5434       * Added anonymously donated WIN32 sbrk emulation
       
  5435       * Malloc, calloc, getpagesize: add optimizations from Raymond Nijssen
       
  5436       * malloc_extend_top: fix mask error that caused wastage after
       
  5437         foreign sbrks
       
  5438       * Add linux mremap support code from HJ Liu
       
  5439 
       
  5440     V2.6.2 Tue Dec  5 06:52:55 1995  Doug Lea  (dl at gee)
       
  5441       * Integrated most documentation with the code.
       
  5442       * Add support for mmap, with help from
       
  5443         Wolfram Gloger (Gloger@lrz.uni-muenchen.de).
       
  5444       * Use last_remainder in more cases.
       
  5445       * Pack bins using idea from  colin@nyx10.cs.du.edu
       
  5446       * Use ordered bins instead of best-fit threshhold
       
  5447       * Eliminate block-local decls to simplify tracing and debugging.
       
  5448       * Support another case of realloc via move into top
       
  5449       * Fix error occuring when initial sbrk_base not word-aligned.
       
  5450       * Rely on page size for units instead of SBRK_UNIT to
       
  5451         avoid surprises about sbrk alignment conventions.
       
  5452       * Add mallinfo, mallopt. Thanks to Raymond Nijssen
       
  5453         (raymond@es.ele.tue.nl) for the suggestion.
       
  5454       * Add `pad' argument to malloc_trim and top_pad mallopt parameter.
       
  5455       * More precautions for cases where other routines call sbrk,
       
  5456         courtesy of Wolfram Gloger (Gloger@lrz.uni-muenchen.de).
       
  5457       * Added macros etc., allowing use in linux libc from
       
  5458         H.J. Lu (hjl@gnu.ai.mit.edu)
       
  5459       * Inverted this history list
       
  5460 
       
  5461     V2.6.1 Sat Dec  2 14:10:57 1995  Doug Lea  (dl at gee)
       
  5462       * Re-tuned and fixed to behave more nicely with V2.6.0 changes.
       
  5463       * Removed all preallocation code since under current scheme
       
  5464         the work required to undo bad preallocations exceeds
       
  5465         the work saved in good cases for most test programs.
       
  5466       * No longer use return list or unconsolidated bins since
       
  5467         no scheme using them consistently outperforms those that don't
       
  5468         given above changes.
       
  5469       * Use best fit for very large chunks to prevent some worst-cases.
       
  5470       * Added some support for debugging
       
  5471 
       
  5472     V2.6.0 Sat Nov  4 07:05:23 1995  Doug Lea  (dl at gee)
       
  5473       * Removed footers when chunks are in use. Thanks to
       
  5474         Paul Wilson (wilson@cs.texas.edu) for the suggestion.
       
  5475 
       
  5476     V2.5.4 Wed Nov  1 07:54:51 1995  Doug Lea  (dl at gee)
       
  5477       * Added malloc_trim, with help from Wolfram Gloger
       
  5478         (wmglo@Dent.MED.Uni-Muenchen.DE).
       
  5479 
       
  5480     V2.5.3 Tue Apr 26 10:16:01 1994  Doug Lea  (dl at g)
       
  5481 
       
  5482     V2.5.2 Tue Apr  5 16:20:40 1994  Doug Lea  (dl at g)
       
  5483       * realloc: try to expand in both directions
       
  5484       * malloc: swap order of clean-bin strategy;
       
  5485       * realloc: only conditionally expand backwards
       
  5486       * Try not to scavenge used bins
       
  5487       * Use bin counts as a guide to preallocation
       
  5488       * Occasionally bin return list chunks in first scan
       
  5489       * Add a few optimizations from colin@nyx10.cs.du.edu
       
  5490 
       
  5491     V2.5.1 Sat Aug 14 15:40:43 1993  Doug Lea  (dl at g)
       
  5492       * faster bin computation & slightly different binning
       
  5493       * merged all consolidations to one part of malloc proper
       
  5494          (eliminating old malloc_find_space & malloc_clean_bin)
       
  5495       * Scan 2 returns chunks (not just 1)
       
  5496       * Propagate failure in realloc if malloc returns 0
       
  5497       * Add stuff to allow compilation on non-ANSI compilers
       
  5498           from kpv@research.att.com
       
  5499 
       
  5500     V2.5 Sat Aug  7 07:41:59 1993  Doug Lea  (dl at g.oswego.edu)
       
  5501       * removed potential for odd address access in prev_chunk
       
  5502       * removed dependency on getpagesize.h
       
  5503       * misc cosmetics and a bit more internal documentation
       
  5504       * anticosmetics: mangled names in macros to evade debugger strangeness
       
  5505       * tested on sparc, hp-700, dec-mips, rs6000
       
  5506           with gcc & native cc (hp, dec only) allowing
       
  5507           Detlefs & Zorn comparison study (in SIGPLAN Notices.)
       
  5508 
       
  5509     Trial version Fri Aug 28 13:14:29 1992  Doug Lea  (dl at g.oswego.edu)
       
  5510       * Based loosely on libg++-1.2X malloc. (It retains some of the overall
       
  5511          structure of old version,  but most details differ.)
       
  5512 
       
  5513 */
       
  5514 
       
  5515