LIDAR Magazine

Should Smart Pixels Store Data—or Simply Reference It?

Qassim Abdullah’s article on Smart Pixels in LIDAR Magazine prompted me to view the concept from a different perspective. Having spent more than twenty years working with building imagery and computer vision applications, I found myself asking a different question from the one I believe the original article was addressing. Rather than asking how much information a Smart Pixel should contain, I began asking what role a Smart Pixel should play in connecting imagery to the knowledge associated with a building.

For purposes of this article, a Smart Pixel is an image pixel that can be related to a location in the physical world and to information associated with that pixel’s location.

Imagine a facility manager looking at an image within a Digital Twin ten years after a building was completed. She points to a fire sprinkler and asks, “Tell me everything you know about this sprinkler.” The answer may include its manufacturer, inspection history, maintenance records, location of the shutoff valve, replacement parts, warranty, work orders, and observations created years after the image was captured.

Should all of that changing information be stored in the image? Or should the image provide a durable way to find whatever is currently known about the physical location it depicts?

This article argues for the second approach. A Smart Pixel should not become a miniature repository. It should provide a persistent reference between an image location and a physical location, allowing knowledge about that location to evolve independently.

Here, information architecture means the division of responsibilities among imagery, location references, and the systems that manage building knowledge. The question is less about how Smart Pixels should be implemented. It is what job they should perform within a larger information ecosystem.

Two Ways of Looking at the Same Image

Photogrammetry and computer vision often begin with the same photographs, but historically they have emphasized different questions.

Photogrammetry has traditionally focused on recovering scene geometry from multiple images and accurately determining the position and orientation of the camera at the moment of image acquisition.

Computer vision, while sharing many of the same geometric foundations, has often been driven by a different objective: understanding the content of an image and relating what is seen to objects and locations in the real world.

These perspectives are complementary rather than competing.

One asks, “Where was the camera?”

The other asks, “What does this pixel represent?”

As building imagery becomes part of long-lived operational records, that second question becomes increasingly important.

SmartPixel Image Aug 4 2026 08 53 51 AM

Camera Pose Is Not Pixel Position

Knowing the camera position and orientation establishes the origin and direction of a viewing ray. It does not, by itself, identify the XYZ location where that ray intersects the physical scene.

In photogrammetry, that location is commonly calculated by intersecting rays from multiple photographs whose camera poses are known. In computer vision, it may be determined from a single image when the pixels have associated depth, lidar, or other real-world measurements. The method may differ, but the architectural point is the same: camera pose and image coordinates are not themselves the physical location represented by the pixel.

That distinction matters because building owners and facility managers rarely ask about camera pose or pixel’s positions. They ask where a shutoff valve is located, what was behind a wall before it was enclosed, when a component was installed, or what has changed since the previous inspection.

The Architectural Observation

Discussions of Smart Pixels can easily turn into questions about how much information should be attached to each pixel. Storage is inexpensive, processors are fast, and networks continue to improve. The tempting answer is to make each pixel contain more.

But the more important question is whether a pixel should be responsible for storing information that will continue to change throughout the life of a building.

Equipment is replaced. Walls are remodeled. Inspections are performed. Maintenance records accumulate. New photographs document construction, commissioning, repairs, and renovations. Artificial intelligence may add observations that did not exist when an image was captured.

The collection of imagery continues to grow, while each individual image remains a record of a particular moment. The knowledge associated with what appears in that image may change for decades.

This suggests a simple separation of responsibilities:

  • Images document what was visible at a particular time.
  • Information systems maintain the changing knowledge associated with the building.
  • Smart Pixels connect image locations to physical locations so that the two can remain linked.

Under this model, a Smart Pixel does not need to know everything. It needs to provide a dependable path to the systems that do.

The Visual Memory of a Building

A completed Digital Twin may use only a fraction of the imagery collected during construction and operation. Yet owners frequently need to look backward.

  • What was behind this wall before the drywall was installed?
  • Was this valve present in the original installation?
  • Has this crack changed since the last inspection?
  • What wiring was installed before the ceiling was closed?

The answers may be found in earlier imagery rather than in the current representation of the building. A large project can therefore accumulate trillions of pixels across construction, renovations, inspections, and operations without requiring every application to load the entire archive at once.

Viewed individually, those photographs are records. Viewed together, they become the visual memory of the building. Persistent location references can allow a current Digital Twin to reach back into that history when earlier conditions matter.

Why This Matters to Owners

Owners rarely ask for photographs as an end product. They ask questions. Imagery supplies visual context; facility management, asset management, and maintenance systems supply current operational knowledge. Neither replaces the other.

A Smart Pixel that consistently references a physical location can connect those systems without requiring the image to duplicate their contents. The image can remain an intact historical record while the associated information continues to grow.

This is why the distinction is more than a storage issue. It is an information architecture for preserving the usefulness of building imagery throughout the operational life of the asset.

Future Work

This article is intended as an introduction to an information architecture, not as a technical specification. Future work should address how persistent location references are created, how uncertainty is represented, and how different capture technologies are reconciled. Those questions belong in a discussion among the photogrammetry, computer vision, reality-capture, facility management, and Digital Twin communities.

The immediate question is more basic: Should Smart Pixels attempt to store the knowledge associated with a building, or should they provide enduring references that allow that knowledge to evolve elsewhere?

I hope this perspective contributes to the continuing discussion of Smart Pixels and the documentation communities who are shaping the future of digital twins.

A Final Thought

The first generation of Smart Pixels may be remembered for making imagery interactive. The next generation may be remembered for connecting imagery to the evolving history of the physical world.

Perhaps the smartest Smart Pixel is not the one that knows everything. It is the one that always knows where to find the truth.

Exit mobile version