Unfortunately the improved lighting introduced with the Orange Box was not added to hammer’s default compile settings, even as options, so we must add them ourselves. That means we need to know what they are and what they do. There are three options that we’re going to look at today, texture shadows, per vertex lighting for static props, and exact outline shadow casting for static props. So what exactly do they do?

-staticproppolys

-staticproppolys tells VRAD to use the exact outline of a static prop when it works out how light is cast. You might think it does this already as I once did but there are some props that cast odd shadows. VRAD uses collision meshes by default because, for many reasons, collision meshes have a lot simpler geometry which makes calculating lighting around them a lot quicker. For realistic lighting though, this is a problem as a lot of props have collision meshes that don’t precisely mimic the contours of the object itself.

-textureshadows

-textureshadows does what it says on the tin, it makes VRAD calculate shadows from textures with alpha channels that are applied to props. Chainlink fences and foliage are the most common props with alpha textures, to save having to model all the individual links in the fence a single sheet is used that has a transparent texture applied to it, but if VRAD is using the collision mesh or even the exact outline of the prop for shadowing then it thinks the links are as solid as a sheet of steel. -Texture shadows solves this. The prop must however be named in a lights.rad file in the root of the game you are compiling for otherwise it won’t bother with it. I’ll cover that in a bit.

-staticproplighting

-staticproplighting calculates per-vertex lighting for static props. Before all static props were lit by a single calculation at their origin or lighting origin. If that was inside a wall the prop was black, if it was in shade the entire prop was dark, if it was in light the entire prop was bright. Now though if half a model is in light and half is in shade, only the vertices that are in light are bright and if part of the prop is inside a wall then only the vertices that are actually inside the wall are black.

Unlike the previous two, this will increase the filesize of a map as well as the compile time. Since the previous two are lightmap based, the resulting filesize is the same with or without them but -staticproplighting forces VRAD to calculate hundreds if not thousands more bits of lighting information for each static prop used in the map.

Lets look at some examples:

This one shows notable improvement for each compile parameter added. For -staticproppolys the shadow cast by the crane is turned from a dark blobby mass into an accurate network of dark and light as the light shines through gaps in the planks of wood. The dark blob around the rocks disappears as VRAD uses the outlines of the rocks instead of the collision mesh that the rocks don’t actually have. The razor wire top of the fence also cast a shadow before but with -staticproppolys the shadow from it is almost nonexistent.
With -textureshadows the massive dark area caused by the fence disappears. If the sun spread angle was lower and the lightmap scale on the ground low enough you’d be able to see the chainlink pattern cast as a shadow here.
Finally -staticproplighting has a subtle effect but if you look carefully at each prop you’ll see it working. The spigot is half shadowed and half lit now, the boulder is casting a shadow onto the smaller part of it that is hidden from the sun, the inside of the concrete tubes are dark now.

This example shows most improvement with -staticproplighting. The corrugated metal sheets have some shadowing on them, the smallest one’s origin is either inside the plank of wood in front of it, or just behind it, in a dark shadow, but with -staticproplighting light falls on all the vertices except for a single strip through the middle. The oil drums have more natural lighting in the final shot, the one in the front casts a shadow on the one behind. You can also see the shadow cast by the fence disappear in the second two screenshots.

How do we use these features?

First you’ll need to enable expert mode on your compile options (click the ‘Expert…’ button on the ‘Run Map’ window) pick a configuration you wish to modify, generally these options are good for very final compiles, I only use them when doing a final build or for fine lighting tweaking, so ‘HDR Full compile -final (slow!)’ is probably a good configuration to alter. Find the $light_exe one in the list of compile commands and select it. There is an image below, but yours won’t look exactly like this because I’ve added a few of my own which I can toggle on/off for different compiles. Add this to the Parameters of the $Light_exe one you just selected (without the quotes) “-lights mylights.rad -staticproppolys -textureshadows -staticproplighting”. Add them at the start of the list, NOT the end. The very final bit in the Parameters box should always be “-game $gamedir $path\$file” which should already be there, just make sure anything you add to $light_exe is in front of -game.

There is one last thing that needs to be done, you need to create a mylights.rad file. This file is just a list of all the props you’re using that need -textureshadows enabling for them. You use ‘forcetextureshadow’ followed by the filepath of the model, each model on a new line. “forcetextureshadow models/props_gameplay/security_fence512.mdl” for example.
Once you have that file, and have added in your parameters in hammer, just hit ‘Go!’ and wait for your compile.

Wait, there’s more!

Whilst these options are generally fantastic for improving visuals, there are a few cases where they make things worse and need to be turned off on a prop by prop basis. There are three options at the bottom of the keyvalues for any static prop which alter how these compile settings work. Well actually, just how -staticproppolys works. ‘Disable Vertex lighting’ does what it says on the tin, stops all per-vertex calculations for the prop in question and uses the pre-2007 method of using just the prop’s origin. ‘Disable Self-Shadowing with vertex lighting’ doesn’t turn off the per-vertex lighting calculations, but rather tells it to ignore other parts of the same prop when calculating lighting. I haven’t found a case where I’ve needed this, but the fact that it’s there means Valve did. Finally ‘Ignore surface normal for computing vertex lighting’ makes each calculation unaware of which direction the face is facing.

Here the top image shows the scene compiled with -textureshadows, -staticproppolys and – staticproplighting turned on but you can see there’s a bit of blotchy lighting on the concrete tube. This is caused by a single vertex on the tube being in exactly the wrong place, it’s in the shadow of one of the chain links. This isn’t a terrible example but if you wanted to remove this unrealistic blotchyness you might want to change the ‘Disable vertex lighting’ option to ‘yes’ which is shown below. You also lose the darkness on the inside of the tube, so you have to weigh the pros and cons in each case. The tube’s properties look like this:

Below is another example that occurs frequently. Single sided models, that is models which are only a single sheet of polygons with a texture using $nocull so you can see the back of it, only receive light from their front. This means if a piece of corrugated metal, such as the centre piece below, is facing away from the main light source it will be unnaturally dark especially if it’s next to similar props facing the opposite direction.

You can see with the ignore surface normals option changed to yes the middle sheet of metal now receives light like the others do. Again, this is what the properties of that prop look like:

Enjoy. (No doubt I’ve made something unclear or have made a mistake, so… comments!)

Some things you might want to consider doing in that options menu you probably have ignored largely:

• Turn off verticies – Your 2D view now only draws the verticies of the brush you have selected, the 2D views are a lot less cluttered this way, especially when zoomed out a lot.
• Default to 15 degree rotations – Using alt to rotate in .5 degree rotations, you can quickly rotate something exactly or roughly depending on what you want.
• Arrow keys nudge selected object/vertex – Very useful for moving stuff by a few increments, even duplication as shift+arrow key will duplicate the object and put the new one one gridline in that direction.
• Increase the distances – The Back clipping plane and model render distances both stop you seeing a vista as it was intended in your 3D view. I have mine well over 8000 units away.
• Uncheck ‘Animate models’ – particularly if you’re working with large animated models such as cinematic explosions the 3D and 2D views can become obstructed, not to mention its just another thing taking time to render.
• Match the camera’s FoV with what you use in game – It goes without saying that a miss match will lead you to view the same area differently in game and in hammer, make sure they match!
• Increase the camera speed – Mine is set to 6500. Holding space turns the left mouse into a freelook and using WASD keys to move around I can move around at great speed.

Use Cordon wisely. See that little hazard stripe button? It isn’t just a compile tool, its far more powerful than that. Cordon can cut through all the mess in your 2D view to show you exactly the area you’re working on. You don’t want to look through your entire level just to see one staircase in detail, just drag a cordon area around it, enable it and you can see your staircase, free of clutter. Like this:

Here the same staircase is viewed, without cordon enabled the staircase is almost invisible behind the rest of the level.

Got two screens? Wondered about hammer’s dual screen capabilities? Well its possible with another thing available in the options menu. “Use independent window configurations” and “Load default window positions with maps” in the general tab, check both. Drag hammer to the size you wan it (It won’t go fullscreen over two monitors unfortunately). Then use Window>New window to add single windows, resize them and position them how you want. Then hit the Save window state button (looks like a stack of windows with an S next to it on the top toolbar). Now whenever you open a map in hammer it will load with this window configuration.

You can also undock and resize the side toolbars, I have mine floating between the two screens and I shifted my top toolbar to the middle too. Here is my display:

I favour the topdown view out of the 2D views, so it got to be the biggest. The fullscreen 3D view accurately represents what I might see ingame after I changed the FoV to match my ingame settings. I added two black bars to the middle so you can see where my screen splits.

I’ve heard people rave about the VBCT’s options for compiles and how easy they are to use. I quite like hammer’s own compile options, if you press ‘Expert…’ on the standard compile options you get to a much more in depth version, I’ve hijacked the HDR full compile configuration and added in a load of my own, I now have a checkbox system for fast/normal vis and normal/fancy lighting, as well as a -makedevshots checkbox option. Unchecking all but bsp is my leak-check compile.

Checkbox options for all the types of compiles I run, easy to pick and choose.

Last but not least, a feature of Windows 7 that I’ve grown particularly fond of: Jump lists. Valve have been quick to take advantage of them and the steam jump list is one of my favourites. You can access jump lists by either right clicking or a click+drag upwards from a taskbar icon. Once opened, you can launch the SDK directly instead of opening the steam window and tools tab. It only saves a few clicks but I like it

Common tasks within steam.

And to round everything off: Learn the shortcuts for all the tools, they’re listed here: Two posts ago.

Take Half-life 2, for example.  You know those combine force-fields, that ripple with energy, and fade out as you move away from it?  Or in Team Fortress 2, you’ll have seen the rotating radar screen in the 2fort basement.  Both those effects are done with material proxies.

Force fields and computer screens animated with material proxies

For this post, I’ve prepared four materials that provide a gradual introduction to several different ways of using proxies.  Since proxies are all about animated textures, screenshots alone won’t be very informative, so I’ve made a “Proxy Museum” HL2 map with these examples in. Download it and run HL2 in a window so you can follow along the article and see the results alongside.  The first example is in the first room, on the left-hand wall.

Download the proxy_museum example map

Let’s start with a simple example. Here is the VMT for a basic texture, a section of carpet:

    "LightmappedGeneric"
    {
    	"$basetexture" "overlays/redrunner001b"
    }

One of the easiest things to do with proxies is to make a scrolling texture, with the “TextureScroll” proxy. Here’s the same texture, only now it is a moving piece of infinite carpet:

    "LightmappedGeneric"
    {
    	"$basetexture" "overlays/redrunner001b"

        "Proxies"
        {
            "TextureScroll"
            {
                "textureScrollVar" "$basetexturetransform"
                "textureScrollRate" 0.25
                "textureScrollAngle" 90
            }
        }
    }

This carpet scrolls endlessly.

The Valve developer wiki has a good reference for many of the proxies available; here’s what it says about TextureScroll:

TextureScroll

Returns a transform matrix or vector that will translate a texture at a given angle at a given rate.

• textureScrollVar — Destination for the resulting transformation.
• textureScrollRate — Rate of scroll in units per second.
• textureScrollAngle — Angle of rotation to move along. (90 = up, 180 = left, etc)

So just by adding these few lines, we now have a texture that scrolls vertically, scrolling one full length in four seconds (0.25 units per second). The most important bit here is the value we give for textureScrollVar.  All proxies need to store their output somewhere; here we are telling it to store its output into the $basetexturetransform property.

If you’re familiar with material properties, you’ll know that $basetexturetransform allows you to scale, rotate, and offset the texture; for example, setting “$basetexturetransform” “center .5 .5 scale 1 1 rotate 0 translate 0 .5″ directly in your VMT would cause the texture to appear shifter vertically by half its size. However by using a proxy, the basetexturetransform is updated for every frame that’s rendered, producing a very smooth scroll.

This picture constantly rotates.

Now turn around, and look at the picture opposite.  Here you have the classic impressionist portrait, which portrays the sitter as merely a tiny part of the endless circle of life.  I’m sorry, I mean it shows a constantly rotating material.

Here’s the VMT for this material:

    "LightmappedGeneric"
    {
        "$basetexture" "models/props_c17/bladeblur"

        "$angle" 0.0
        "$translate" "[0.0 0.0]"
        "$center" "[-0.5 -0.5]"

        "Proxies"
        {
            "LinearRamp"
            {
                "rate" 45
                "initialValue" 0.0
                "resultVar" "$angle"
            }

            "TextureTransform"
            {
                "translateVar" "$translate"
                "rotateVar" "$angle"
                "centerVar" "$center"
                "resultVar" "$basetexturetransform"
            }
        }
    }

Unlike the previous example, there’s no proxy specifically for continuously rotating textures, so instead we use the TextureTransform proxy:

TextureTransform

Generates a texture transform matrix.

• centerVar — Name of a variable holding the center of rotation and scaling. [Optional]
• scaleVar — Name of scale variable (2D vector). [Optional]
• rotateVar — Name of rotation angle variable (float). [Optional]
• translateVar — Name of the translation variable (2D vector).
• resultVar — Resulting value.

This proxy lets you set the texture’s scale, rotation, and translation all at once, but instead of giving it values directly, now we need to create variables, and give the variable names to TextureTransform.  To create a variable, just give it a name (beginning with $) and a starting value, before the Proxies section:

        "$angle" 0.0
        "$translate" "[0.0 0.0]"
        "$center" "[-0.5 -0.5]"

The type of the variable depends on the value; $angle is just a number, while $translate and $center are 2-dimensional vectors.  $center will be the center of rotation, wile $translate is how much to offset the texture.  We don’t want to offset it at all actually, but TextureTransform insists that you give a value for “translateVar”.

If that was all we had, the texture would be nicely rotated by zero degrees; so we need to make the angle always increasing so the texture rotates continuously.

LinearRamp

Produces an ever increasing value.

• rate — Rate of increase of the value.
• initialValue — Initial value to increase.
• resultVar — Resulting value.

LinearRamp to the rescue: by adding the LinearRamp proxy, we make $angle increase by 45 degrees every second.

The picture disappears as you look at it.

Around the other side of this central wall is an interesting picture, that you cannot easily see—it fades away when you look at it directly.  Can you guess how this was done?  That’s right, proxies!  Here’s the VMT for the material:

    "LightmappedGeneric"
    {
        "$basetexture" "Metal/metalwall078a"
        "$surfaceprop" "metal"

        "$envmap" "env_cubemap"
        "$envmaptint" "[ .56 .56 .75 ]"

        "$bumpmap" "metal/metalwall078a_normal"
        "$nodiffusebumplighting" 1
        "$normalmapalphaenvmapmask" 1

        "$dpscale" 7.0
        "$dotproduct" 0.0
        "$alpha_unclamped" 0.0

        "Proxies"
        {
            "PlayerView"
            {
                "scale" 7.0
                "resultVar" "$dotproduct"
            }
            "Subtract"
            {
                "srcVar1" "$dpscale"
                "srcVar2" "$dotproduct"
                "resultVar" "$alpha_unclamped"
            }
            "Clamp"
            {
                "min" 0.0
                "max" 1.0
                "srcVar1" "$alpha_unclamped"
                "resultVar" "$alpha"
            }
        }
    }

The Valve wiki describes the PlayerView proxy like this:

PlayerView

Stores the dot product of the view direction + the direction from the camera to the entity the material is on.

• scale — Amount to scale the dot product by.
• resultVar — Resulting value.

What all that actually means is that its result will be 1.0 if the player is looking directly at the center of the material, decreasing down to 0.0 if looking at right angles to it, and to -1 when the player is facing away.  Here we scale that by 7.0 to get a result from -7.0 to 7.0 in the $dotproduct variable.

There are a number of proxies for performing mathematical calculations: Add, Subtract, Multiply, Divide, and more.  Here we use two of them:

Subtract

Subtracts the second variable from the first.

• srcVar1
• srcVar2
• resultVar

Clamp

Keeps a variable within a specified range.

• min,
• max — The acceptable range of values
• srcVar1
• resultVar

We subtract $dotproduct from seven so it will range from zero when looking directly at the material to fourteen when looking away, and store the result in $alpha_unclamped.

Finally, since the $alpha value is expected to be between zero and one.  Fourteen is obviously well out of that range, so we clamp $alpha_unclamped so that it’s never less than zero or more than one, and store the result in $alpha.  The end result is that the material is now fully transparent when looking at the center of it, becomes opaque as the player looks away.

The tint of the material is animated randomly.

The final picture in the gallery shows Dr. Breen in his true colours—all of them.  Taking inspiration from Andy Warhol, sixteen func_brushes with portraits of Dr. Breen are arranged in a mosaic.  The same material is applied to each of them, the variation is controlled by proxies:

    "LightmappedGeneric"
    {
    	"$basetexture" "scripted/breen_fakemonitor_"

        // The base texture will be multiplied by this color
        "$color" "[1.0 1.0 1.0]"

        "$offset" "0"

    	"Proxies"
    	{
            "EntityRandom"
            {
                "scale" "12"
                "resultVar" "$offset"
            }
            "Sine"
            {
                "sineperiod" 4
                "timeoffset" "$offset"
                "resultVar" "$color[0]"
            }
            "Sine"
            {
                "sineperiod" 8
                "timeoffset" "$offset"
                "resultVar" "$color[1]"
            }
            "Sine"
            {
                "sineperiod" 12
                "timeoffset" "$offset"
                "resultVar" "$color[2]"
            }
    	}
    }

Here we are manipulating $color, a standard material property which is a vector of red, green, and blue values that tints the texture.  This material modifies the red, green, and blue values independently to vary the tint, with three sine waves.

Sine

Creates a sine wave.

• sineperiod — Period of the sine wave in seconds.
• sinemin — Minimum value of the sine wave.
• sinemax — Maximum value of the sine wave.
• timeoffset — Used to start the sine wave at a different point in time
• resultVar — Resulting value.

The Sine proxy does exactly what it says on the tin: creates a sine wave.  The sinemin and sinemax paramters default to 0 and 1 if not set.  So the three Sine proxies we use are creating 3 sine waves with periods of 4, 8, and 12 seconds respectively.

Another important point to note here: the $color material property is a three-element vector.  When outputting values from proxies, using square brackets with a number will set just a single element of a vector.  So the first Sine proxy in this material outputs to “$color[0]“, changing the red value only.

The other proxy in this material is quite important for the effect. If we just had the three Sine proxies, all the sixteen portraits would animate together.  So we want each of them to start its sine waves from a different point, for which we use the EntityRandom proxy:

EntityRandom

A proxy that returns a random number associated with the entity the material is applied to. Can be helpful for making animated textures not all be in sync.

• scale — Amount to scale the random number by.
• resultVar — Resulting value.

Using this, each brush will animate starting at a different point in the 12-second cycle, producing the final effect that you see.

Material proxies allow you to do some very cool effects.  This article has just scratched the surface of what you can do with them.  If you want to learn more, read the Valve developer wiki’s page on proxies and its list of proxies for guidance, but to really learn you’ll need to study existing examples, and experiment for yourself.

Three practical examples of material proxy use.

If you walk into the remaining room of the museum, you’ll find three more displays: each of these shows the use of material proxies to achieve a practical effect.

The waves are animated with proxies.

Ocean waves: Three sine waves and numerous calculations are used to animate simple waves breaking on the shore.  The wave moves into the shore at a slight angle, washes up, then washes away, fading quickly.

The TV screens flicker and jump.

Combine TV: A staticky TV screen image that constantly jumps as if reception is poor.  This uses two GuassianNoise proxies to generate random numbers to control the jumping of a base texture, and a UniformNoise proxy to randomly offset the second texture of the UnlitTwoTexture shader.

The laser beams flicker and fade.

Chess, with Lasers: A UniformNoise proxy makes the lasers flicker constantly, while the PlayerProximity proxy is used with calculations to subtly fade the lasers when viewed from a distance or very near.

The VMF of the map, and VMTs of all the materials can be downloaded here; hopefully these will inspire you to create your own amazing effects!

1. Plan while not at hammer. Idle time spend at your keyboard adds up; know what you want to do before you sit down. All your time in hammer should be spent at maximum speed.
2. Compile very infrequently. These take too much of your time that you could otherwise be spending working. Complete your daily task sheet and compile at the end of the day; take note of issues ingame and prepare to fix those tomorrow.
3. Cordon compiles: Don’t lazily seal your map; do it methodically and strongly according to optimization practices. Until then, use the cordon method to test your compiles.
4. Use your keyboard shortcuts wisely and swiftly! Proper hammer hand position is not on ASDF but on SHIFT Z X C V. Use your X finger to shift up to S. You should not be wasting time pressing buttons with the mouse, when you can do it in microseconds with shortcuts.

• Shift: When you grab a brush, hold shift and drag it to duplicate the brush. Don’t bother making a new brush, this wastes your time! Grab the nearest, closest shaped brush and duplicate it into place
• Shift+S: In a tool mode, but want to go back to selection? Shift+S hops you into the selection mode. CTRL Q releases your current selection.
• Shift+X: Have a brush you need to split? Shift+X brings up the cut tool. Press again to cycle through modes.
• Shift+V: Vertex editing; this should be your #1 used tool. Press again to cycle through modes.
• CTRL C: Copy, duh! But use it with right click -> Paste Special to duplicate large selections onto the original’s position.
• [ and ]: Use these to quickly alter grid size.
• CTRL B: Snap to grid!
• SHIFT A: Bring up the texture applicator.
• ALT ENTER: Brings up entity window.
• SHIFT O: Overlays
• CTRL+T ties to entity, usually func_detail. CTRL W will bring it back to world.
• Shift when rotating takes you off 15 degree increments for fine tuning.
• CTRL+M allows you to scale selections, rotate and more.

5. Set up your displacements properly. A little vertex manipulation here will save you time down the road.
6. Paint displacements face normal. Face normal is by default on the Z axis; alt+right click on a face or displacement vertex to shift the plane of displacement alteration. This will vastly speed up your displacement painting.
7. Use viewgroups wisely. Turn off tool textures whenever you’re not working with them; likewise brush ents can often get in the way. Remove everything but world brushes when optimizing and func_detailing.
8. When working with symmetrical maps, center an enormous skip brush around the center of your map; you can then duplicate pieces of one side and select the brush to rotate them cleanly to the correspondent area on the other half of the map. CTRL L flips horizontally for bilaterally symmetric maps.
9. Make use of your fellow mappers’ tools! These can eliminate painful processes that take too long but have to be often done. Examples:

• Boojum’s Prefab pack and Gametype gallery. The prefabs within can help you set up doors, spawns, CPs and other entity arrangements that would normally take some time to put together.
• Acegikmo’s environment gallery. Allows you to quickly experiment with exterior lighting arrangements.
• Ravidge’s lighting gallery. Allows swift implementation of manmade lighting.

Full List of Hammer Shortcuts

This tutorial assumes you have an intermediate understanding of using hammer and the terminology used for its processes.

Part I: Visibility in Source

For this exercise we will be considering the source game TF2, though these techniques will apply to source in general. Our players will consist of a brick wall, an engineer representing the player, and a sentry representing the high-polygon props that end up creating framerate loss on our players’ end. Our goal today will be to instruct the engine not to draw the sentry when it is indeed hidden by the brick wall.

We would assume that, when looking at the wall, the engineer’s GPU would not be rendering the sentry. In source, not necessarily so! On compile, when vbsp is run, the engine makes imaginary cuts out of any open space to determine what players should render. Confused? Well, let’s look at the scene from above.

All entities are ignored in this “cutting”, which we call a “binary space partition.” Sound familiar? The initials, bsp, are found in our maps’ file type. Binary space partition sounds like a complicated phrase, but is easily illustrated by this example taken from Wikipedia:

The BSP process cuts the interior of your map into pieces, one by one, until each separate piece is no longer concave. Then, the visibility calculator (vvis.exe) imagines itself as each of these created pieces, and attempts to draw lines outwards to every other piece. If, from any position in one piece it has a line of sight to another, it considers that piece visible. Let’s look at it in context. Remember that world brushes, here drawn in red, make our square concave, so the engine will cut around it, in green:

Now let’s attempt to draw lines from the engineer’s partition to the sentry’s partition.

As you can see, even from the deepest angle, we can’t draw a line from the engy’s space to the sentry’s. Therefore, the engy won’t render the sentry! Hoorah!

But what if the engine had instead, arbitrarily, partitioned the scene starting across the wall‘s longer sides, like this?:

As you can see, the created partitions are all convex and therefore valid, but look at this:

We can easily draw a line from the engy’s partition to the sentry’s, and so he would render it while looking at the wall! This is clearly incorrect, so how do we tell the engine to partition it like our first example, rather than this inferior second one? The answer lies within the tool texture, Hint. We use hint brushes to tell the BSP system- or rather, to hint to it- where it should perform its partitions first. A hint brush is constructed by texturing a world brush entirely with skip, and then applying the hint tool material along the face where we want a cut made. Let’s quickly add hints to our example so that it partitions correctly. We want to place them so that those cuts along the wall’s skinny edges are made first.

This is now correctly hinted, and presuming our scene had four walls and a ceiling in addition to the brick wall, would be cut as in our first partition example. So what can we take from this?

Part II: Optimization Structures

Hints

We will now go into a discussion of practical uses of hints. From here on out, we will be using cordons, marked in red, to represent the walls and ceilings surrounding our scene; red lines will represent world brushes, and purple represent the hint faces; yellow lines will denote possible lines of sight out of any one partition.

Our scene was actually one of the simpler uses of the most basic structure of optimization- the T structure. The T structure is formed by a wall perpendicular to another wall, along whose front a hint brush is placed. In hammer, the structure would look like so:

This is the simplest of optimization situations, and one that can be combined into many other derivative forms. The first of these derivatives is the H structure, which we’ve studied: A solitary wall bracketed by two hint brushes.

In this situation we’ve replaced one wall with a hint brush, so that players on each side of the hints can see down the hall, but those within the width of the wall cannot see the sentry.

We can combine this with a few T-like walls to create something more feasible:

We now have 4 Ts jutting out from our cordon walls to touch the hint brushes. This means that players in the halls can see both areas, but any players in the partitions behind any of the 3 walls on the engineer’s side cannot see into the sentry’s partition. We’ll call this shape the ]-[ configuration.

The next shape, the S shape, is created out of 2 individual Ts. It is frequently used within tunnels to prevent line of sight issues and help optimize in this fashion:

We’ve added another pair of engie and sentry to demonstrate the dual quality of this simple arrangement. The bottom engie cannot see either sentry, and the top engie can see 1 but not both.

We can add in a third T to create a more solid optimization corridor, in the W shape:

This should look familiar, as the W is basically half of our ]-[ shape, but designed to work with only a single path instead of two paths of travel.

If we rotate the W to be on a diagonal and then snap it to the grid, we come upon another oft-used optimization shape, one that is used in many corners. We’ll call it the L shape. First, we rotate our W:

Now we vertex manipulate it to come out with a more familiar pathway:

As you can see, we now have a right angle corridor, but even though our walls have changed shape the W’s optimization is still at work. Only players inside of the green triangle can see both areas. We can use the L shape in a variety of places. An easy example is the right-most corridor on 2fort as you enter the enemy base from the outside second-floor; it leads to their spawn room area.

It is important to note in all of these cases that we have been assuming a solid world brush ceiling. This will usually not be the case in your maps! But luckily, the T structure can be used in horizontal situations as well, and combined with all of the other shapes to form their ceiling. Only players below the horizontal hint will benefit from the vertical optimization shapes, so make sure that if players will be above the horizontal hint, it will only be shortly. A simple use:

Our T is still intact, perpendicular to the floor; in fact, in this position you can see where it derives its name. Let’s see it in combination with the W:

So while there is no ceiling abutting our shorter walls, the T function of the horizontal hint allows the W to still work correctly. It is important to note that without a ceiling or horizontal hint, there is no telling how vbsp and vvis will interpret your optimization structure! If any of the partitions extends above the walls involved with the particular optimization structure, VVIS will be able to see down from the top of the partition into other partitions, and players in that first partition will therefore render objects otherwise hidden by walls.

Part III: Examples of Hint Structures in Use

We have now covered all of the basic optimization shapes that involve hints. Experiment with combining Ts into other new structures to complement your map design. OPTIMIZATION MUST BE CONSIDERED WHEN CONSTRUCTING A LAYOUT. These structures are simple but need to be in mind when laying out a map! Remember that any of these structures can be scaled to any size, and we will demonstrate macro uses of these micro shapes in this next section: real examples. I’ll be using mostly my own work to point it out; you may download them if you wish to see these in game. Our actors will be represented by two blue circles; in each scenario the circles will not render one another.

Basic T:

Here’s an example from Yukon’s CP1 area of a Basic T in action. This demonstrates how large a scale you can use it on, and how you can vary the thickness of the T’s trunk:

Basic H:

Again from Yukon, this is the upper transition from CP2 to CP3:

You can see how large you can expand these structures to be without interrupting function.

S Shape: Atrophy, near B and A, we have an example of a larger S-shape. Blue cannot see one another, and red cannot see one another; additionally, the top left blue cannot see the bottom right red:

W Shape: The W is one of my favorite; it can be implemented on an enormous scale like here in cp_atrophy, between B and C:

L Shape: This is easy to implement in simple fashion, but very cool to use in a different manner like here in Yukon, in the alternate path between CP3 and CP2 (cliffs removed for visibility):

Here’s another even larger example in Atrophy between A and C:

Vertical T: These are everywhere, but a simple example is in ctf_aurora, where a horizontal hint covers the entirety of the map and guarantees that players in the backyards cannot see those in the middle:

There are unlimited examples of such structures being twisted and used in every imaginable fashion, so keep an eye out! I couldn’t possibly cover each instance.

Part IV: Other Optimization Brushes, a Primer

Area Portals

Now we will cover, in short, areaportals in their door function. Area portals are brush entities, completely covered in the areaportal material and tied to a func_areaportal. Area portals act like world brushes when closed, and like open space (with some restrictions) when open. We use area portals within doors to prevent players from drawing anything inside the doored area. However, the area behind the area portal must be its own ‘area’ in the sense that, other than through another area portal, no VVIS lines (our yellow arrows) can be drawn into a partition on the outside of the area. A simple way to test this is to start from any visible face of your area portals and try to navigate through empty space to another face of the same area portal. If you’re able to do so without entering another area portal, you need to seal the area better! But if you cannot, then your area is well-sealed and ready for compile. Area portals throw leaks if not sealed correctly, and can be a pain to sort out, but that’s too indepth for our brief overview here. Let’s look at a simple case of using an area portal:

Here we have our original scenario, but with the wall extended to fit against our cordon bounds and a func_door added right between our two actors. Remember that a door is an entity and therefore is excluded from the vbsp process; it cannot by itself block visibility. This is where the area portal comes in: we place it within the door such that it is entirely enclosed and touching all 4 faces (3 brick, 1 floor) around the door. We then set its properties so:

If your door starts closed, as would ours, we also start the area portal as closed. We then set its linked door to be the name of our door. Now, when the door’s position is toggled, the area portal will toggle between open and close: once the door is fully closed, our area portal will shut and act like a world brush and the engineer will stop rendering the sentry. These are best used with spawn doors and other doors that will only be open momentarily; this gives you a higher budget (budget in optimization terms is the amount of detail you can use without impacting framerates negatively) to work with, without using one of our hint structures (which tend to complicate areas) instead.

Occluders

There is one final brush entity we need to know when optimizing (in TF2; other games use area portal windows, which we can explain at a different time.) This entity is called a func_occluder. It acts as a curtain between you and props on its opposite side; each side of an occluder entity textured with the occluder tool material will prevent you from rendering any objects completely covered by that face. Let’s take a situation where we have a func_detail wall instead of a world brush wall. It’s ignored in visibility calculations, but we still want our player not to render the sentry. Here is our scenario:

In this case we don’t want players at the sentry’s position to render the engy either, so we’ll make an occluder with two faces textured with Occluder. Like a hint brush, its other faces will be skip. We create this brush either within the wall or exactly as big as the wall:

We’ve toggled off details in our visgroups so that we can see the Occluder here. Now, neither the sentry nor the engy, were they both players, would be able to render one another. Occluders sound great, but they’re not at always a good idea! It costs the client to calculate which props should be ignored, so be sure that each occluder prevents a large number of high poly props from being drawn to balance out the initial cost of calculation. Look to Gravelpit’s vmf for correct usage.

Part V: Other Ways to Optimize

This concludes our overview of brush-based optimization, but there are a few other ways to optimize we need to keep in mind. These are kind of like cheating, and you’ll only squeeze scant frames out of each, so be sure to use hints and all before you go through and do these:

1. Draw distances. You can assign fading properties to any props that will prevent them from being drawn at a certain distance away. This works best on small detail props, but beware that it costs the client to fade the prop, so they will only experience benefits when the prop is completely invisible.
2. Func_lod. This is an entity that is exactly like a func_detail except that it has fade distances like props. Use these sparingly.
3. Lightmap optimization. Look it up elsewhere Doesn’t offer any huge gains, but shrinks file size.
4. Func_detail. The less partitions created by vbsp, the shorter vvis takes and the less calculations your players have to perform within each partition. Use func_detail to reduce the complexity of your vbsp structure. To view the structure of your map, compile it and then go to Map->Load Portal File and choose the associated file. The blue lines are the cuts that have been made around each partition!
5. Advanced area portal usage. Area portals have small performance benefits when placed in doorways correctly; again, find a different tutorial for this particular piece.
6. Fog. At the limits of your map’s fog distance, decided by your env_fog_controller, nothing more is rendered; if you’re aggressive with this distance, you can slightly reduce framerate loss. Not advised.
7. Water. You can use cheap water instead of expensive water (I presume you know what these are)
8. Displacement power. If you can reduce the power of your displacements (working with power 2 is pro!) by even 1 degree, you will drastically reduce the energy needed to render that particular face. Remember that power 4 represents 8x the number of vertexes in power 2, 2x power 3. Even going from power 3 to 2 reduces the number of vertices considerably.
9. Nodraw. The engine automatically deletes brush faces facing the void or completely covered by other brush faces, but there are often faces that do not satisfy these conditions that will never be seen by your players- usually the back faces of detail behind player clips. Applying the nodraw material to these faces will remove them from the cost of rendering.

Part VI: Closing, Extra Links, and Glossary

We have looked at a variety of ways to optimize your maps, but there are more details to be learned! Now that you have a basic knowledge, search the VDC and elsewhere for specifics on theory and indepth explanations of any of the aforementioned knowledge areas. I hope you are able to come out with the next great work, and one that will perform well on all machines! Look for combinations of what we’ve examined, and be clever; the more you press yourself to be creative in optimizing, the less of a chore it’ll feel- imagine it a challenge

LINKS:

1. VDC
2. Great guide with more theory
3. Maps used in discussion:

• Atrophy
• Yukon is included within TF2 as cp_yukon_final
• Aurora