Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Colors = "5ae59095-9a9b-59fe-a467-6f913c188581"
ImageBase = "c817782e-172a-44cc-b673-b171935fbb9e"
LazyModules = "8cdb02fc-e678-4876-92c5-9defec4f444e"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
RegionTrees = "dee08c22-ab7f-5625-9660-a9af2021b33f"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"

[compat]
Clustering = "0.14.3"
Expand Down
8 changes: 6 additions & 2 deletions src/ColorQuantization.jl
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
module ColorQuantization

using Colors
using ImageBase: FixedPoint, floattype, FixedPointNumbers.rawtype
using ImageBase: FixedPoint, floattype, FixedPointNumbers.rawtype, FixedPointNumbers.N0f8
using ImageBase: channelview, colorview, restrict
using Random: AbstractRNG, GLOBAL_RNG
using LazyModules: @lazy
using RegionTrees
using StaticArrays

#! format: off
@lazy import Clustering = "aaaa29a8-35af-508c-8bc3-b662a17a0fe5"
#! format: on
Expand All @@ -15,8 +18,9 @@ include("api.jl")
include("utils.jl")
include("uniform.jl")
include("clustering.jl") # lazily loaded
include("octree.jl")

export AbstractColorQuantizer, quantize
export UniformQuantization, KMeansQuantization
export UniformQuantization, KMeansQuantization, OctreeQuantization

end
154 changes: 154 additions & 0 deletions src/octree.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@

struct OctreeQuantization <: AbstractColorQuantizer
numcolors::Int
function OctreeQuantization(
colorspace::Type{<:Colorant},
numcolors::Int = 256;
kwargs...,
)
colorspace == RGB{N0f8} || error("Octree Algorithm only supports RGB colorspace")
return new(numcolors)
end
end

const OCTREE_DEFAULT_COLORSPACE = RGB{N0f8}
# Color to RGB
# constructor for struct
function OctreeQuantization(numcolors::Int = 256; kwargs...)
return OctreeQuantization(OCTREE_DEFAULT_COLORSPACE, numcolors; kwargs...)
end

function (alg::OctreeQuantization)(img::AbstractArray)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried running

using ColorQuantization, TestImages

img = testimage("fabio")
img = RGB{N0f8}.(img)
alg = OctreeQuantization(64)
alg(img)

however, the algorithm doesn't terminate after waiting several minutes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is fabio having high number of colors combined with how I prune the tree, lines using allleaves(root) is where trouble is

julia> unique(img)
37499-element Array{RGB{N0f8},1} with eltype RGB{N0f8}:
 RGB{N0f8}(0.541,0.506,0.51)
 RGB{N0f8}(0.459,0.337,0.325)
 RGB{N0f8}(0.51,0.341,0.306)
 RGB{N0f8}(0.455,0.341,0.278)
 RGB{N0f8}(0.435,0.349,0.306)
 ⋮
 RGB{N0f8}(0.898,0.569,0.424)
 RGB{N0f8}(0.906,0.58,0.416)
 RGB{N0f8}(0.925,0.596,0.424)
 RGB{N0f8}(0.89,0.561,0.388)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually horrifyingly slow ;-;

return octreequantisation!(img; numcolors = alg.numcolors)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the other quantisation methods, calling the quantizer directly (or calling quantize(img, alg)) currently returns a colorscheme and not the quantized image.
This is why other methods don't have in-place modifying versions and why ColorQuantization currently doesn't have a quantize! function.

We'll have to brainstorm possible API designs for different use cases:

  • A): image to colorscheme (current master)
  • B): image to quantized image (this PR?)
  • C): image to "fitted" quantizer. This quantizer could then
    • take a color and return the closest quantized color
    • take an image and quantize it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would want to do this and reluctant to provide API for third given incase we do return a function that acts as quantizer, that will compare each pixel with all quantized colors for similarity which might make it unusable but for a single color it would work:

img, palette = quantize(img, alg; return_palette=true)
img = quantize(img, alg; return_palette=false)
quantize!(img, alg) # we can return palette here too but looks unusual

end

function octreequantisation!(img; numcolors = 256, precheck::Bool = false)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

precheck should be made part of the OctreeQuantization struct, otherwise it will be inaccessible to users.

Minor nitpick: To keep the spelling consistent with the American English package name, "quantization` should be spelled with a "z".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Do we always want to run precheck then? There is a big added cost to precheck's unique call
  • Sure thing I'll update the quantisation with quantization

# ensure the img is in RGB colorspace
if (eltype(img) != RGB{N0f8})
error("Octree Algorithm requires img to be in RGB colorspace")
end
Comment on lines +15 to +17
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe octreequantization could cast input images to RGB{N0f8} for the user instead of deepcopying them. This way, this error could be avoided automatically.

Copy link
Member Author

@ashwanirathee ashwanirathee Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could surely but then we are quantising a different image entirely in a different colorspace, we def could avoid deepcopying if we are not gonna accept it if it not right type


# checks if image has more colors than in numcolors
if precheck == true
unumcolors = length(unique(img))
# @show unumcolors
if unumcolors <= numcolors
@debug "Image has $unumcolors unique colors"
return unique(img)
end
end

# step 1: creating the octree
root = Cell(
SVector(0.0, 0.0, 0.0),
SVector(1.0, 1.0, 1.0),
["root", 0, [], RGB{N0f8}.(0.0, 0.0, 0.0), 0],
)
cases = map(
p -> [bitstring(UInt8(p))[6:end], 0, Vector{Int}([]), RGB{N0f8}.(0.0, 0.0, 0.0), 1],
1:8,
)
split!(root, cases)
inds = collect(1:length(img))

function putin(root, in)
r, g, b = map(p -> bitstring(UInt8(p * 255)), channelview([img[in]]))
rgb = r[1] * g[1] * b[1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are strings used here to represent UInt8 colors?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't use it as a way to identify color but as a identifier of case number of which possible cases are ["000","001"...."111"] eight different cases of octree at each level. More details on this can be found here: http://delimitry.blogspot.com/2016/02/octree-color-quantizer-in-python.html

Copy link
Member Author

@ashwanirathee ashwanirathee Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Albeit I do agree that using rgb = r[1] * g[1] * b[1] and then doing root.children[i].data[1] == rgb is misleading

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But now that I think about it, do we even need the bitstring?.......

# finding the entry to the tree
ind = 0
for i = 1:8
if (root.children[i].data[1] == rgb)
root.children[i].data[2] += 1
ind = i
break
end
end
curr = root.children[ind]

for i = 2:8
cases = map(
p -> [
bitstring(UInt8(p))[6:end],
0,
Vector{Int}([]),
RGB{N0f8}.(0.0, 0.0, 0.0),
i,
],
1:8,
)
rgb = r[i] * g[i] * b[i]
if (isleaf(curr) == true && i <= 8)
split!(curr, cases)
end
if (i == 8)
for j = 1:8
if (curr.children[j].data[1] == rgb)
curr = curr.children[j]
curr.data[2] += 1
push!(curr.data[3], in)
curr.data[4] = img[in]
return
end
end
end

# handle 1:7 cases for rgb to handle first seven bits
for j = 1:8
if (curr.children[j].data[1] == rgb)
curr.children[j].data[2] += 1
curr = curr.children[j]
break
end
end
end
end

# build the tree
for i in inds
root.data[2] += 1
putin(root, i)
end

# step 2: reducing tree to a certain number of colors
# there is scope for improvements in allleaves as it's found again n again
leafs = [p for p in allleaves(root)]
filter!(p -> !iszero(p.data[2]), leafs)
tobe_reduced = leafs[1]

while (length(leafs) > numcolors)
parents = unique([parent(p) for p in leafs])
parents = sort(parents; by = c -> c.data[2])
tobe_reduced = parents[1]
# @show tobe_reduced.data

for i = 1:8
append!(tobe_reduced.data[3], tobe_reduced.children[i].data[3])
tobe_reduced.data[4] +=
tobe_reduced.children[i].data[4] * tobe_reduced.children[i].data[2]
end
tobe_reduced.data[4] /= tobe_reduced.data[2]
tobe_reduced.children = nothing

# we don't want to do this again n again
leafs = [p for p in allleaves(root)]
filter!(p -> !iszero(p.data[2]), leafs)
end

# step 3: palette formation and quantisation now
da = [p.data for p in leafs]
for i in da
for j in i[3]
img[j] = i[4]
end
end

colors = [p[4] for p in da]
return colors
end

function octreequantisation(img; kwargs...)
img_copy = deepcopy(img)
palette = octreequantisation!(img_copy; kwargs...)
return img_copy, palette
end