> Duke Nukem as well used a BSP engine
The Build engine didn't use BSP, it treated connections between sectors as portals and rasterized the walls as (90 degree rotated) trapezoids while performing clipping against those portals. This allowed it to have dynamic wall geometry (e.g. moving trains, rotating light fixtures, etc) as well as "room-over-room" setups as long as you couldn't see both rooms at the same time (in both Blood and Shadow Warrior they found a workaround for it allowing to create more "3D" spaces by making identically shaped sectors with the floor of one sector acting as a portal to the ceiling of the other sector - supposedly this wasn't "natively" supported by the engine, but it was flexible enough for the game studios who used it -without even having access to the source- to do it themselves).
The first level of Duke Nukem 3D does use a few Build tricks - e.g. another one is that sprites can be "axis aligned" instead of following the camera and they can also have collision - this can be used to create rudimentary 3D geometry by treating each sprite as an axis aligned quad and in the first level it is used to make a bridge between two buildings (right before the level exit button).
Though it must be said that Duke's flexibility came with a tradeoff - while BSPs will find a leaf node in log(n) time, no such guarantee exists for Duke and its up to the mappers to optimize the maps so that the renderer doesn't need to traverse a large amount of sectors.
I saw recently on the website of the guy who built the Build engine that licensees got some .c files and some .o files (with the rough breakdown being game code in .c files and engine code in .o files) but I guess if you knew enough you could hack around.
I always loved that the bridge you mentioned could take damage and fall down, screwing you over in the very first level, unless you knew where the Jetpack was stashed.
The funny thing is that looking backwards, I would never use a grid of squares for a raycaster like wolfenstein3d did.
If I were to do a raycaster today, I would use convex sectors with portals, basically like duke nukem, but constant wall heights. You can do drawing very simply by just doing a linear pass across the sector, recursively stepping into other sectors.
Then you can at least do arbitrary level geometries.
Yeah, that is basically what i did around 2009 when i was making a 3D engine in Haxe for Flash, RayFaster 2 (it was the successor to another Flash game engine, RayFaster, which did Wolf3D-like raycasting and i used to make a simple FPS[0] though with Flash's input limitations i couldn't do mouselook and didn't feel that great).
The engine used Build/Doom-like maps and would simply raycast against the camera's current sector (sectors could be any shape not just convex) and used connections between sectors as portals to recursively follow the (2D) map until it hit a solid wall - so basically Build's portal rendering approach except using raycasting instead of trapezoid rasterization. It could also do sprite rendering (you could have both sprites facing the camera and "aligned" sprites that could be used for decals) and even had some simple 3D model rendering (the triangles were sorted and clipped against the portal during rasterization). Unfortunately Flash isn't available in browsers anymore and Ruffle isn't compatible with it (it can run the engine -much slower than real Flash- but the palette is all wrong), so i took some shots using the standalone Flash "projector"[1][2][3][4]. Also making maps for it was certainly much more laborsome than making maps for a Wolf3D-like game and that combined with me losing interest in making Flash games meant i never made a game with it.
[0] https://www.youtube.com/watch?v=Z81NhEbl3q8
[1] http://runtimeterror.com/pages/iv/images/01feb493af6dd3184b8...
[2] http://runtimeterror.com/pages/iv/images/136a8753b211ed7e3d1...
[3] http://runtimeterror.com/pages/iv/images/84c2012258982b82053...
[4] http://runtimeterror.com/pages/iv/images/e94cc334c7735e1aacb...
A grid of squares makes sense for the target hardware at the time though (286 or better CPU)
It really doesn't. I would argue that convex sectors with portals could be quicker than casting rays across a grid for every pixel column, except for some degenerate cases. This is because for any pixel column in-between the boundaries of wall segments, there's virtually no work to be done (O(1)), compared to the work of casting a ray along the grid O(n).
Btw, 286 played wolfenstein rather poorly. The 386 is rather more appropriate.
Wolfenstein was built in a couple of months by a then 21-year-old Carmack. He didn't focus on optimizing levels until Doom.