An all-optical arithmetic unit with the help of terahertz-optical-asymmetric-demultiplexer (TOAD)-based tree architecture is proposed. We describe the all-optical arithmetic unit by using a set of all-optical multiplexer, all-optical full-adder, and optical switch. The all-optical arithmetic unit can be used to perform a fast central processor unit using optical hardware components. We have tried to exploit the advantages of both optical tree architecture and TOAD-based switch to design an integrated all-optical circuit that can perform binary addition, addition with carry, subtract with borrow, subtract (2’s complement), double, increment, decrement, and transfer operations.