|
Understanding the "3GB not 4GB" Memory Limitation in Windows XP
and Vista
PART 1:
Question: I just bought a system with 4GB of physical RAM in it. The
BIOS posts 4GB, but Windows tells me that I have anywhere from 2.75 -
3.5GB of RAM. Where is the rest of my RAM?
Summary: If you are running 32-bit Windows, you must live with it. You
will not ever see all 4GB of RAM but it WILL be used by the system for
hardware devices. If you are running 64-bit Windows, you may or may see
all the memory, depending on your motherboard's chipset, your system may
support memory remapping. If so, you will be able to see all 4GB of RAM.
Due to an architectural decision made long ago, if you have 4GB of
physical RAM installed, Windows is only able to report a portion of the
physical 4GB of RAM (ranges from ~2.75GB to 3.5GB depending on the
devices installed, motherboard's chipset & BIOS).
This behavior is due to "memory mapped IO reservations". Those
reservations overlay the physical address space and mask out those
physical addresses so that they cannot be used for working memory. This
is independent of the OS running on the machine.
Significant chunks of address space below 4GB (the highest address
accessible via 32-bit) get reserved for use by system hardware:
• BIOS – including ACPI and legacy video support
• PCI bus including bridges etc.
• PCI Express support will reserve at least 256MB, up to 768MB depending
on graphics card installed memory
What this means is a typical system may see between ~256MB and 1GB of
address space below 4GB reserved for hardware use that the OS cannot
access. Intel chipset specs are pretty good at explaining what address
ranges gets reserved by default and in some cases call out that 1.5GB is
always reserved and thus inaccessible to Windows.
When looking at memory in systems (be it desktop or notebook) there are
three questions to ask that will tell you the maximum amount of memory
your O/S will be able to use:
1. What O/S Edition have you installed?
a. 32-bit Windows is limited to a maximum of 4GB and cannot see any
pages above 4GB.
b. 64-bit Windows can use between 8GB and 128GB depending on SKU.
2. What address range can your processor actually access?
a. Typically that’ll be 40-bit addressing today for x64 (Intel
EM64T/AMD64), but older processors may be limited to 36-bit or even
32-bit
3. Can your system’s chipset map memory above 4GB?
a. Mobile chipsets on sale today cannot (but that may change with time)
b. Newer workstations (which use chipsets developed for single
or multi-proc servers) usually can.
Windows can remap memory from below 4GB to above 4GB and use it there,
however, that relies on the three points above:
1. Can Windows access memory above 4GB?
a. 32-bit – NO
b. 64-bit – Maybe (due to chipset limitations)
2. Can your processor access memory above 4GB?
a. If it’s recent then it might, and if it’s either AMD64 or EM64T
(Intel 64-Bit) it’s almost certain
3. Does your chipset allow pages to be remapped above 4GB?
a. Probably not – and that’s what’s catching people who install 64-bit
Vista to work around point 1 – they find they still cannot see above 4GB
In some cases, OEMs may be able to tweak their BIOS to reserve less
memory for platform use, but we’re not talking a huge difference (ie,
100’s of MBs). In the end a 32-bit OS and/or application can only,
ever, handle 4GB of memory at a time, the AWE stuff just swaps chunks of
memory in and out of that 4GB space, thus fooling the application and OS
into using more space than it can “see”.
PART TWO:
To be perfectly clear, this isn't a Windows problem-- it's an x86
hardware problem. The memory hole is quite literally invisible to
the CPU, no matter what 32-bit operating system you choose. The
following diagram from Intel illustrates just where the memory hole is:
TECHNICAL: (source: author Brian Madden 2004)
There seems to be a lot of confusion in the industry about what's
commonly called the Windows “4GB memory limit.” When talking about
performance tuning and server sizing, people are quick to mention the
fact that an application on a 32-bit Windows system can only access 4GB
of memory. But what exactly does this mean?
By definition, a 32-bit processor uses 32 bits to refer to the
location of each byte of memory. 2^32 = 4.2 billion, which means a
memory address that's 32 bits long can only refer to 4.2 billion unique
locations (i.e. 4 GB).
In the 32-bit Windows world, each application has its own “virtual”
4GB memory space. (This means that each application functions as if it
has a flat 4GB of memory, and the system's memory manager keeps track of
memory mapping, which applications are using which memory, page file
management, and so on.)
This 4GB space is evenly divided into two parts, with 2GB dedicated
for kernel usage, and 2GB left for application usage. Each application
gets its own 2GB, but all applications have to share the same 2GB kernel
space.
This can cause problems in Terminal Server environments. On Terminal
Servers with a lot of users running a lot of applications, quite a bit
of information from all the users has to be crammed into the shared 2GB
of kernel memory. In fact, this is why no Windows 2000-based Terminal
Server can support more than about 200 users—the 2GB of kernel memory
gets full—even if the server has 16GB of memory and eight 3GHz
processors. This is simply an architectural limitation of 32-bit
Windows.
Windows 2003 is a little bit better in that it allows you to more
finely tune how the 2GB kernel memory space is used. However, you still
can't escape the fact that the thousands of processes from hundreds of
users will all have to share the common 2GB kernel space.
Using the /3GB (for Windows 2000) or the /4GT (for Windows 2003)
boot.ini switches is even worse in Terminal Server environments because
those switches change the partition between the application memory space
and kernel memory space. These switches gives each application 3GB of
memory, which in turn only leaves 1GB for the kernel—a disaster in
Terminal Server environments!
People who are unfamiliar with the real meaning behind the 4GB
Windows memory limit often point out that certain versions of Windows
(such as Enterprise or Datacenter editions) can actually support more
than 4GB of physical memory. However, adding more than 4GB of physical
memory to a server still doesn't change the fact that it's a 32-bit
processor accessing a 32-bit memory space. Even when more than 4GB of
memory is present, each process still has the normal 2GB virtual address
space, and the kernel address space is still 2GB, just as on a normal
non-PAE system.
However, systems booted /PAE can support up to 64GB physical memory.
A 32-bit process can "use" large amounts of memory via AWE (address
windowing extension) functions. This means that they must map views of
the physical memory they allocate into their 2GB virtual address space.
Essentially, they can only use 2GB of memory at a time.
Here are more details about what booting /PAE means from Chapter 7 of
the book "Inside Windows 2000," by David Solomon and Mark Russinovich.
All of the Intel x86 family processors since the Pentium Pro
include a memory-mapping mode called Physical Address Extension (PAE).
With the proper chipset, the PAE mode allows access to up to 64 GB of
physical memory. When the x86 executes in PAE mode, the memory
management unit (MMU) divides virtual addresses into four fields.
The MMU still implements page directories and page tables, but a
third level, the page directory pointer table, exists above them. PAE
mode can address more memory than the standard translation mode not
because of the extra level of translation but because PDEs and PTEs are
64-bits wide rather than 32-bits. The system represents physical
addresses internally with 24 bits, which gives the x86 the ability to
support a maximum of 2^(24+12) bytes, or 64 GB, of memory.
As explained in Chapter 2 , there is a special version of the
core kernel image (Ntoskrnl.exe) with support for PAE called
Ntkrnlpa.exe. (The multiprocessor version is called Ntkrpamp.exe.) To
select this PAE-enabled kernel, you must boot with the /PAE switch in
Boot.ini.
This special version of the kernel image is installed on all
Windows 2000 systems, even Windows 2000 Professional systems with small
memory. The reason for this is to facilitate testing. Because the PAE
kernel presents 64-bit addresses to device drivers and other system
code, booting /PAE even on a small memory system allows a device driver
developer to test parts of their drivers with large addresses. The other
relevant Boot.ini switch is /NOLOWMEM, which discards memory below 4 GB
and relocates device drivers above this range, thus guaranteeing that
these drivers will be presented with physical addresses greater than 32
bits.
Only Windows 2000 Advanced Server
and Windows 2000 Datacenter Server are required to support more than 4
GB of physical memory. (See Table 2-2.) Using the AWE Win32 functions,
32bit user processes can allocate and control large amounts of physical
memory on these systems
|
