sensors

Article
A Computer Vision-Based Roadside Occupation
Surveillance System for Intelligent Transport in
Smart Cities
George To Sum Ho 1 , Yung Po Tsang 2 , Chun Ho Wu 1, * , Wai Hung Wong 1 and
King Lun Choy 2
1 Department of Supply Chain and Information Management, The Hang Seng University of Hong Kong,
Shatin, Hong Kong, China; [email protected] (G.T.S.H.); [email protected] (W.H.W.)
2 Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hunghom,
Hong Kong, China; [email protected] (Y.P.T.); [email protected] (K.L.C.)
* Correspondence: [email protected]; Tel.: +852-3963-5550




Received: 14 March 2019; Accepted: 12 April 2019; Published: 15 April 2019


Abstract: In digital and green city initiatives, smart mobility is a key aspect of developing smart cities
and it is important for built-up areas worldwide. Double-parking and busy roadside activities such
as frequent loading and unloading of trucks, have a negative impact on traffic situations, especially
in cities with high transportation density. Hence, a real-time internet of things (IoT)-based system
for surveillance of roadside loading and unloading bays is needed. In this paper, a fully integrated
solution is developed by equipping high-definition smart cameras with wireless communication
for traffic surveillance. Henceforth, this system is referred to as a computer vision-based roadside
occupation surveillance system (CVROSS). Through a vision-based network, real-time roadside traffic
images, such as images of loading or unloading activities, are captured automatically. By making
use of the collected data, decision support on roadside occupancy and vacancy can be evaluated by
means of fuzzy logic and visualized for users, thus enhancing the transparency of roadside activities.
The CVROSS was designed and tested in Hong Kong to validate the accuracy of parking-gap estimation
and system performance, aiming at facilitating traffic and fleet management for smart mobility.

Keywords: smart mobility; computer vision; roadside occupation; traffic surveillance; smart city

1. Introduction
Traffic congestion is a persistent problem worldwide, leading to economic and social challenges.
To enhance competitiveness, smooth traffic conditions are of the utmost importance for any city. This
is especially true when developing a smart city, which aims at making good use of information and
communication technologies (ICT) to support the creation of a ubiquitous and interconnected network
of citizens and organizations, sharing digital data and information via the internet of things (IoT) [1].
In recent years, smart mobility and smart transportation have been advocated to alleviate the above
social and public problems.
In general, frequent roadside loading and unloading activities have a negative impact on traffic
situations. An increasing number of vehicles and insufficient data transparency regarding roadside
activities, occupancy and vacancy, make the situation worse, and this issue is becoming critical in
cities with high transportation density. For instance, during peak hours, limited roadside spaces are
usually occupied by trucks for loading or unloading or by other vehicles for purposes of picking up
or dropping off. Some vehicles are required to circle the road network or temporarily double-park
while waiting for roadside spaces to become available. Figure 1 illustrates a real-life situation of
roadside double-parking, caused by the loading and unloading activities of logistics trucks, company

Sensors 2019, 19, 1796; doi:10.3390/s19081796 www.mdpi.com/journal/sensors

surveillance of roadside loading and unloading bays, namely, a computer vision-based roadside
occupation surveillance system (CVROSS). By equipping a set of wireless, high-definition vision
devices with machine learning and solar power in a vision-based network, real-time roadside traffic
images – including recognition of loading or unloading activities – can be observed and captured
automatically, to enhance the transparency of roadside occupancy and vacancy. Decision support
Sensors 2019, 19, 1796 2 of 26
models for parking-gap calculations and parking-space assignment are formulated, to eliminate the
occurrence of double-parking, while fuzzy logic is applied to address the fuzziness in vehicle parking
reservations
cars and otherand estimation
obstacles. Theseof actions
the time of stay
cause for vehicles.
unnecessary As a and
blocking result, the safety
create proposed system
issues can
for other
evaluate
road average
users, spaceflow
and traffic utilization, loading Hence,
can be affected. and unloading
there areactivities
increasing and averageofwaiting
numbers times for
applications
parking. surveillance
roadside Property management companies
systems, especially withand drivers
respect can derive
to traffic benefits
surveillance relating
related to time-saving
to increasing traffic
and smoother
congestion. traffic flow
However, in busy
current built-up
roadside areas. To validate
surveillance systems the proposed
have system’s
limitations, such performance
as being affectedand
feasibility,
by a case study
vehicle speeds was
[2]. To be conducted in the Kwun
useful, a roadside Tong District
surveillance system ofrequires
Hong Kong (one of
machine the busiest
learning and
districts for
computer roadside loading
vision-based and to
technology unloading
facilitate activities
and enhancein Hong Kong). This
the accuracy enabled the formulation
and effectiveness of detection
of proactive
and recognition strategies,
of vehiclesimproving
and objectsnot
[3].only
Withefficiency in transportation
an increasing number of vehiclesand intraffic flowsituations
roadside but also
roadside
which maysafety
cause for thetraffic
severe public. A paireda sample
congestion, suitable t-test was applied tosystem
and well-functioning evaluate the hypothesis
is needed, utilizing
regarding
ICT and IoT thetechnologies.
difference between estimated
This would enableparking gaps (generated
surveillance of roadsidefrom the CVROSS)
loading and unloadingand actual
bays,
parking
to prevent gaps, socongestion
traffic that the performance
and facilitateoftraffic
the CVROSS
and fleet could
planningbe validated.
and managementIn addition, satisfaction
by implementing
and system
smart mobility, performance
hence achieving werea highly
evaluated by conducting
efficient road networka insurvey
a smartofcity.
drivers and property
Furthermore, smart
management company
transportation and smart representatives
parking can bewho use, andunder
established are responsible
the ontologyfor of
managing, roadside areas.
smart mobility.

Figure
Figure On-siteroadside
1.1.On-site roadside example
exampleofofdouble-parking.
double-parking.
In an attempt to solve these problems, this paper presents an IoT-based system for the
surveillance of roadside loading and unloading bays, namely, a computer vision-based roadside
occupation surveillance system (CVROSS). By equipping a set of wireless, high-definition vision
devices with machine learning and solar power in a vision-based network, real-time roadside traffic
images—including recognition of loading or unloading activities—can be observed and captured
automatically, to enhance the transparency of roadside occupancy and vacancy. Decision support
models for parking-gap calculations and parking-space assignment are formulated, to eliminate the
occurrence of double-parking, while fuzzy logic is applied to address the fuzziness in vehicle parking
reservations and estimation of the time of stay for vehicles. As a result, the proposed system can
evaluate average space utilization, loading and unloading activities and average waiting times for
parking. Property management companies and drivers can derive benefits relating to time-saving
and smoother traffic flow in busy built-up areas. To validate the proposed system’s performance and
feasibility, a case study was conducted in the Kwun Tong District of Hong Kong (one of the busiest
districts for roadside loading and unloading activities in Hong Kong). This enabled the formulation of
proactive strategies, improving not only efficiency in transportation and traffic flow but also roadside
safety for the public. A paired sample t-test was applied to evaluate the hypothesis regarding the
difference between estimated parking gaps (generated from the CVROSS) and actual parking gaps,
so that the performance of the CVROSS could be validated. In addition, satisfaction and system
performance were evaluated by conducting a survey of drivers and property management company
representatives who use, and are responsible for managing, roadside areas.
Sensors 2019, 19, 1796 3 of 26

This paper is organized as follows: Section 2 contains a literature review related to this study.
In Section 3, the architecture of the CVROSS is presented, demonstrating how it can be applied in Hong
Kong, known as one of the busiest cities for road usage. Section 4 presents a case study, implementing
the CVROSS to validate its feasibility and performance. Section 5 introduces the results and discusses
the findings, together with their implications. Finally, conclusions are drawn in Section 6.

2. Literature Review
In this section, an overview of roadside surveillance systems is given. The methods and emerging
IoT technologies used in roadside surveillance systems are then discussed, to define the research gap
addressed by this paper.

2.1. Overview of Roadside Surveillance Systems

With the advancement of technology, roadside surveillance systems have developed significantly
in terms of object detection, tracking, classification and behaviour analysis, thus improving accuracy and
reliability. Surveillance refers to the processes of focusing systematic and routine attention on certain
human behaviours for influencing, managing, protecting or directing purposes [4]. This may entail
observation from a distance by means of electronic equipment, such as closed-circuit television cameras
(CCTV) or interception of electronically transmitted information, such as internet traffic or phone calls.
Surveillance is frequently utilized by governments for intelligence gathering, crime prevention and
investigation, or for the protection of a process, person, group or object. For example, traffic surveillance
by video cameras is common, using cameras installed in many different locations, e.g., mounted along
the highways to monitor real-time traffic situations, thus controlling traffic flow and facilitating traffic
management by governments. According to Valera and Velastin [5], roadside surveillance systems
are utilized to observe the roadside activities and behaviours of vehicles and road users, and they
include moving object detection, recognition, classification, tracking, behavioural analysis and retrieval.
This enables real-time monitoring of persistent and transient objects within a specific environment.
By applying advanced information and communication technologies, roadside surveillance systems
are able to solve transport problems such as road hazards and traffic congestion and, ultimately,
they can help to achieve transport efficiency [6]. Smooth traffic conditions are crucial to road users and
the public, and traffic congestion can lead to a range of negative effects. According to Robinson [7],
traffic congestion is very costly and has an obvious negative effect on productivity, fuel consumption,
environmental quality and quality of human life. The effects of traffic congestion include, but are
not limited to, delays in journeys, increasing air pollution and carbon dioxide emissions, blocking of
emergency vehicles and spillover effects from congested roads to other roads. For instance, traffic
congestion may delay the schedules of truck drivers’ loading and unloading activities, thus affecting the
management of logistics companies. Some vehicles even circle the road network to wait for available
parking spaces. This wastes fuel and energy. Therefore, the ultimate objective of roadside surveillance
systems is to facilitate traffic and fleet management, since traffic congestion, accidents and low traffic
efficiency result in a waste of resources [8].

2.2. Methods of Roadside Surveillance Systems

Video surveillance has drawn the attention of scientists and engineers to active application-oriented
research on computer vision, artificial intelligence and image processing. This is a contributing factor
in the increasingly widespread deployment of intelligent video-based surveillance systems [9]. One of
the most significant applications of intelligent video-based surveillance systems, in which a growing
interest has emerged in recent years, is roadside traffic surveillance for dealing with problems of
increasing traffic congestion. Current techniques include vehicle detection and tracking to measure
traffic parameters and car park management. Traffic conditions are of crucial importance for road users
and society generally. Thus, various applications of roadside surveillance systems aim at utilizing
image processing methods to obtain better traffic conditions and maintain a high level of road safety,
Sensors 2019, 19, 1796 4 of 26

thus achieving a more efficient traffic network [10]. The development of roadside surveillance systems
for the measurement of traffic parameters has been a focus of research in recent years. Data on the
number of vehicles and their type, speed and flow, are collected by roadside surveillance systems using
vehicle detection and tracking techniques. This contributes to transportation planning, traffic operation
and pavement design. According to Huang [11], roadside cameras have been applied to estimate
lane boundaries and vanishing points, hence classifying vehicles by their physical length, to monitor
traffic situations on the roads via video image processing techniques. In addition, Coifman et al. [12]
designed a real-time computer vision system for vehicle tracking and traffic surveillance by considering
the lighting conditions in daylight and twilight and at night. Saran and Sreelekha [13] also utilized a
video-based surveillance system to measure traffic parameters. The system used could be divided into
three main functions: detection, classification and tracing of vehicles. This also enabled the number of
vehicles to be counted. To provide real-time analytical information, computer vision techniques were
used to process images. With the traffic information produced (traffic congestion, number of vehicles,
classification, etc.), the video-based surveillance system was expected to achieve and maintain efficient
traffic management and road safety. By considering traffic conditions, related applications mainly
focus on vehicle detection and tracking, processing the data and images using computing techniques.
Hence, useful information about traffic parameters can be produced for further analysis, transportation
planning, traffic operation and pavement design.
In addition to the measurement of traffic parameters, the number of applications of surveillance
systems for facilitating the management of both indoor and outdoor car parks is increasing. In this
type of application, systems are mainly used for counting the number of parked vehicles, monitoring
changes in parked vehicles over time and identifying available parking spaces [14]. Searching for
parking spaces consumes a significant amount of travelling time, regardless of whether it is in a car park
or on the roadside. Moreover, there is always a difficulty in maintaining and providing information
in real time without using an intelligent system. Therefore, in recent years, many researchers have
studied the possibility of using vision-based car park management systems to determine occupancy
and vacancy and to provide users with related information. Lin et al. [15] proposed a vision-based
parking management system to manage an outdoor car park, using four cameras set up in the lofts of
surrounding buildings to record occupancy and vacancy, then processing the data and transferring the
information to users. Greyscale images were captured to maintain colour consistency and increase the
accuracy of the system. Furthermore, Micheloni et al. [16] utilized a surveillance system for car parks
by managing both static and active cameras in a hierarchical framework. The static camera system
employed a variable number of static sensors to maintain the trace of all objects simultaneously, while
the active camera system used a pan-tilt camera to capture high definition (HD) video of the target,
to prevent problems with multi-sensor, multi-target tracking. These surveillance systems applied to
car park management are beneficial for counting the number of parked vehicles, monitoring changes
in the parked vehicles over time and identifying available parking spaces. They are also likely to be
applicable to roadside loading and unloading bays for locating occupancy and vacancy, thus alleviating
the problem addressed in this paper.

2.3. IoT Technologies for Roadside Surveillance

Following recent advances in technology, roadside surveillance is not only based on cameras but
also on other emerging technologies, aimed at achieving various aims and objectives. Researchers have
been most interested in the development and applications of radio tomographic imaging (RTI), wireless
sensor networks (WSNs) and computer vision for roadside surveillance. Radio tomographic imaging
is an emerging technology that localizes and tracks moving physical objects in an area surrounded by
simple and inexpensive radios in wireless networks [17]. The concept of RTI originates from radar
systems. Using concepts from radar, RTI originally worked by placing small and inexpensive radios
around the area of interest. The radios were able to transmit and receive wireless signals, creating
a dense network of links passing through the area. If an object moved within the area, it would
Sensors 2019, 19, 1796 5 of 26

reflect or absorb the wireless signal and prevent some of the power from reaching its destination.
Hence, an image could be formed of where the power was being absorbed, by utilizing the link’s
power-loss measurements. Therefore, it could indicate the locations of any detected and tracked objects.
More recently, RTI has been proposed for tracking the location of objects via radio waves, without
requiring objects to transmit or receive radio signals. This occurs when the position is extracted by
inferring which voxels are obstructing a subset of radio links in a dense wireless sensor network [18].
Anderson et al. [19] demonstrated a novel application of RTI with a secure wireless sensor network
for roadside surveillance and vehicle identification and tracking, by combining frames of a moving
vehicle into a single image using a vehicle class-identification algorithm. However, the technology of
RTI is subject to the speed at which it can detect objects moving through the network [2]. Speed is
limited by the time it takes to perform a single scan of the network area. Vehicles on roads are not
always static. If some vehicles are travelling at high speeds which are out of the system specification
range, RTI might not record anything, thus affecting the process of measuring traffic parameters. Thus,
RTI is not totally suitable for all-round roadside surveillance; only for object detection and tracking at
relatively low speeds.
WSNs use networked microsensor technology, which is one of the most important technologies
of the 21st century and a key technology for the future [20]. Wireless sensor networks offer an
attractive, low-cost alternative to inductive loops, video and radar, for traffic surveillance on freeways,
at intersections and in car parks. According to Cheung et al. [21], these networks typically consist
of a set of sensor nodes comprising a sensor, a microprocessor, a radio, a battery and an access
point, together with a radio and a more powerful processor. The sensor nodes are utilized to process
real-time measurements and transmit useful data to the access point located at the roadside, through
direct communication via either line or solar power. This then enables the detection of vehicles and
the estimation of various traffic parameters. Cheung and Varaiya [22] utilized a network of small
sensor nodes, communicating wirelessly among themselves, to sense the physical world. After the
collection of data from the detection events using the sensor nodes, the access point could calculate
the number, occupancy and speed of monitored traffic. It is evident, therefore, that WSNs offer an
attractive, low-cost alternative to inductive loops, video and radar, for traffic surveillance. However,
Tiwari et al. [23] have argued that WSNs are easily affected by their surroundings, e.g., by walls,
microwaves or signal attenuation due to large distances. These networks also have a comparatively low
speed of communication with the server and are easily disrupted by elements such as Bluetooth. Due to
the instability of communication and concerns about interruption, WSNs may not be appropriate for
providing road users and the public with accurate traffic data and information.
Furthermore, technological improvement has led to the development of semi-automatic systems
(known as computer vision) for creating algorithms by using computers for automatic real-time
object and event detection and to aid recognition [24]. According to Huang [25], from an engineering
perspective, computer vision aims at building autonomous systems and seeks to automate, with greater
efficiency, tasks that are performed by the human visual system. In fact, several monitoring objectives
can be supported by the application of computer vision and pattern recognition techniques, such as
detection of traffic violations and identification of road users [26]. Beymer et al. [27] presented a
feature-based tracking approach for the task of tracking vehicles during periods of traffic congestion.
Video cameras were expected to be mounted on poles or other tall structures looking down at the traffic
scene. Huang and Yen [28] designed a real-time and colour-based computer vision system for traffic
monitoring, by analysing colour image sequences of traffic scenes recorded by mounting a stationary
camera on a tall building or a pedestrian crossing bridge near a traffic light. Although computer
vision-based surveillance systems have various functionalities, such as vehicle detection, recognition
and classification, the systems require relatively long processing times and considerable memory,
since converting collected data into useful information can be challenging. In addition, the set-up for
camera calibration to facilitate the extraction of the required images is difficult and time-consuming [29].
However, according to Zander et al. [30], machine learning can be applied to enhance the accuracy and
Sensors 2019, 19, 1796 6 of 26

Sensors 2019, 19, x 6 of 25

efficiency of detection and recognition of an object, using computer vision-based surveillance systems.
Thus, systems are
programmed, by able to learn
exploring thewithout being explicitly
construction programmed,
of algorithms. by exploring
This facilitates the construction
more efficient calculationof
algorithms. This facilitates
and measurement moreinformation
of real-time efficient calculation and measurement
on roadside of real-time
activities, occupancy, information
vacancy on
and traffic
roadside
parameters.activities, occupancy, vacancy and traffic parameters.
In
In summary,
summary, an an effective
effective roadside
roadside surveillance
surveillance system
system is
is critical
critical for
for maintaining
maintaining road safety,
safety,
alleviating
alleviating traffic
traffic congestion
congestion and and facilitating
facilitating traffic
trafficand
andfleet
fleetmanagement.
management. Applications
Applications for for the
the
measurement
measurement of of traffic
traffic parameters
parameters and and car
car park
park management
management utilizing
utilizing computer
computer vision,
vision, show
show an an
increasing
increasingtrend.
trend.Computer
Computervisionvisiontechnology
technologycan canbebe powerful
powerful when
when properly
properlymatched
matchedwithwith machine
machine
learning
learning and
and big
big data
dataanalysis,
analysis, outweighing
outweighing the the disadvantage
disadvantage of of long
long processing
processing times.
times. In
In addition,
addition,
this
this can
can lead
lead toto better
better analysis,
analysis, with
with aa high
highlevel
levelofofaccuracy
accuracyand andefficiency
efficiency[31].
[31]. Furthermore,
Furthermore,
improvements
improvementsinincamera camera calibration should
calibration be considered
should when utilizing
be considered computer
when utilizing vision, tovision,
computer maintain to
the accuracy,
maintain theefficiency
accuracy,and effectiveness
efficiency of the systems
and effectiveness andsystems
of the to accountandfor
to various
accountfeatures on different
for various features
roads, such asroads,
on different the surroundings and the amount
such as the surroundings andofthe
sunlight.
amount of sunlight.

3.
3. Design
Design of
of aa Computer
Computer Vision-Based
Vision-BasedRoadside
RoadsideOccupation
OccupationSurveillance
SurveillanceSystem
System(CVROSS)
(CVROSS)
The
The system
system design
design and
andmodelling
modelling of
ofthe
theCVROSS
CVROSSare aredescribed
describedininthis
thissection.
section. Figure
Figure 22shows
shows
an
an illustrative overview of
illustrative overview ofthe
theCVROSS
CVROSSdeployed
deployedatat the
the roadside.
roadside. TheThe design
design of the
of the CVROSS
CVROSS has
has four
four major components: (i) roadside surveillance technology using the IoT, (ii) data preprocessing
major components: (i) roadside surveillance technology using the IoT, (ii) data preprocessing in the in
the CVROSS, (iii) decision support for roadside parking and (iv) evaluation of the proposed
CVROSS, (iii) decision support for roadside parking and (iv) evaluation of the proposed system. It system.
Itaims
aimsatatreducing
reducingissues
issuesofofdouble-parking
double-parkingininurban
urban transport
transport systems
systems and
and improving
improving the the visibility
visibilityofof
roadside situations. The transparency of roadside occupancy and vacancy can be further
roadside situations. The transparency of roadside occupancy and vacancy can be further enhanced. enhanced.

Figure2.
Figure Overview of
2. Overview ofthe
theCVROSS.
CVROSS.
3.1. Roadside Surveillance Technology Using the IoT
3.1. Roadside Surveillance Technology Using the IoT
To collect data effectively and efficiently, an IoT-based roadside surveillance system (CVROSS)
To collect data effectively and efficiently, an IoT-based roadside surveillance system (CVROSS)
has been designed and developed to tackle the problem of loading and unloading bays. As shown in
has been designed and developed to tackle the problem of loading and unloading bays. As shown in
Figure 2, the CVROSS was equipped with a set of solar power-enabled wireless HD vision devices,
Figure 2, the CVROSS was equipped with a set of solar power-enabled wireless HD vision devices,
which enable the system to capture images from the roadside. To reduce electrical costs and avoid
which enable the system to capture images from the roadside. To reduce electrical costs and avoid
the risks of relying heavily on external power or solar energy, the devices connect wirelessly to
the risks of relying heavily on external power or solar energy, the devices connect wirelessly to a
a cloud platform, allowing continuous data transfer to the CVROSS and real-time monitoring of
cloud platform, allowing continuous data transfer to the CVROSS and real-time monitoring of
occupancy and vacancy data, retaining up-to-date roadside information for 24 h a day, seven days
occupancy and vacancy data, retaining up-to-date roadside information for 24 h a day, seven days a
a week. By using application program interfaces (APIs), the collected data can be examined at the
week. By using application program interfaces (APIs), the collected data can be examined at the
preprocessing stage by the proposed decision support model; thus, the roadside traffic information
preprocessing stage by the proposed decision support model; thus, the roadside traffic information
can be observed in real time. Via a vision module and machine learning, users are provided with
can be observed in real time. Via a vision module and machine learning, users are provided with
hundreds of functions for acquiring images from a multitude of vision devices, for further processing
hundreds of functions for acquiring images from a multitude of vision devices, for further processing
by locating features, identifying objects and measuring parts. In addition, machines can learn from
empirical data, making predictions about future data. HD vision devices are expected to be used, to
provide the best compromise between maximum observation accuracy and minimum overlapping
Sensors 2019, 19, 1796 7 of 26

by locating features, identifying objects and measuring parts. In addition, machines can learn from
empirical data, making predictions about future data. HD vision devices are expected to be used,
to provide the best compromise between maximum observation accuracy and minimum overlapping
field of view, to generate the best viewpoint. Image data are then processed by denoising and image
tuning, leading to target object detection, recognition, identification, classification and calculation of
available parking spaces. As a result, useful and easily accessible traffic information on real-time
roadside occupancy and vacancy can be provided to road users. Furthermore, with the aid of machine
learning techniques applied in a time-domain dynamic system, both the accuracy and the efficiency of
Sensors 2019, 19, x 8 of 25
the system are enhanced. Various reports can be generated for road users, logistics companies and the
public,
of for decision-making
obstacles, via big
such as a road sign, data analysis.
a traffic cone and yellow box markings. All these were a hindrance
After installation of the wireless HD vision
to vehicle and object recognition and matching. Noise devices connected
reduction to achieved
was the CVROSS, cloud-based
using an “image servers
mask”
to exclude irrelevant regions of the image, “colour plane extraction” to convert the colour function
can be used with IoT communication protocols. Data analytics and computation modules image to asa
the back-end
binary image cloud
in onlyserver,
black and
andthe results
white, andare retrieved
“basic and displayed
morphology” in front-end
to modify the shapeapplications for end
of binary objects
users.
in The process
the image, as wellflow
as toofadjust
the proposed systemAfter
the brightness. is illustrated in Figurethe
noise reduction, 3. indicator,
Transparency ofcone
traffic roadside
and
activities and information can be enhanced and reports can be produced at the
yellow box markings had been removed (blacked out) and only the private car remained on the screen end of every timed
loop,its
with after
shapeimage processing.
shown in white.

Start

Setup of Parameters

Vision Acquisition

Noise Reduction

Vehicle and Object Recognition and Matching

Calculation of Parking Gaps

Calculation of Available Parking Spaces

No
Information Visualization

Time Control

Stop?

Yes

End

Figure 3.3.Process
Figure Processflow
flow of the
the CVROSS.
CVROSS.

3.2. Data Preprocessing in the CVROSS

Before running the CVROSS, parameters must be set up, including types of vehicles, vehicle parking
space regulations and minimum width of traffic lanes. These correspond to the regulations issued by
the Hong Kong Special Administrative Region Planning Department [32]. Therefore, the system can
compare the captured images with templates in the database, in order to process images and data more
accurately in the later stages. To facilitate the calculation of parking gaps and available parking spaces,
differences in the dimensions of all items caused by non-identical distances from the vision device are
ignored in the computation process. In other words, it is assumed that each of the items presented in a
Figure 4. An example of noise reduction.

3.2.2. Vehicle and Object Recognition and Matching

Two common matching methods are provided by the CVROSS: pattern matching and geometric
matching. If all the items that need to be detected and matched share the same features, pattern
matching is the best method, as it will compare all the features and colours of an item from the
Sensors 2019, 19, 1796 8 of 26

case has the same dimensions in millimetres or pixels, regardless of its position (in terms of distance)
in relation to the vision device. In the computation process, the preliminary parameters include:
Sensors 2019, 19, x 8 of 25
• 640 × 480 pixels for the entire coverage of the vision device
• 11 m of regulated
of obstacles, parking
such as a road sign,space per truck
a traffic cone and yellow box markings. All these were a hindrance
•
to vehicle and object recognition and matching.van
7 m of regulated parking space per cargo Noise reduction was achieved using an “image mask”
• 5 m ofirrelevant
to exclude regulated regions
parkingofspace per private
the image, carplane extraction” to convert the colour image to a
“colour
•
binary6.75 m forinthe
image minimum
only black andwidth
white,of and
traffic lanesmorphology” to modify the shape of binary objects
“basic
in thetemplates
• image, asofwell as to adjust
all possible the brightness.
vehicles and objectsAfter noise reduction, the indicator, traffic cone and
yellow
• box markings
a confidence had
score, beenindicates
which removedthe (blacked out) and
confidence only
of the the private
disparity car remained
for each on the
pixel for each screen
template
with(image
its shape shown in white.
scores return values between 0 and 1000, where 1000 indicates the highest confidence).

After the parameters are input into the CVROSS, Start the program starts the first stage of the timed
loop, i.e., vision acquisition. The wireless HD vision devices connected to the CVROSS can then capture
images from the roadside continually and automatically. Afterwards, the collected data are used in:
Setup of Parameters
(i) noise reduction and (ii) vehicle and object recognition and matching.
Vision Acquisition
3.2.1. Noise Reduction
One of the most important stages in the entire Noisesystem
Reductionflow is noise reduction. This is a process of

removing noise from an image, as the noise might degrade both the visual quality and the effectiveness
Vehicle and Object Recognition and Matching
of subsequent processing tasks [33]. In this case (according to the simulation model), on the roadside
and in traffic lanes, there are different objects and signals, such as traffic indicators and instructions in
Calculation of Parking Gaps
traffic lanes. However, these are likely to be unrelated to vehicle and object recognition and matching,
therefore they may negatively affect matching results and the effectiveness of the subsequent calculation
Calculation of Available Parking Spaces
of available parking spaces. Furthermore, even similar vehicles, such as two private cars in this case,
No
may be the same model but different in colour. Therefore, noise reduction can ensure that unrelated
Information Visualization
objects, indicators and signals are removed before further processing of the images. This also prevents
problems with colour classification. In Figure 4,Time an example
Control
of noise reduction is illustrated. Before
noise reduction, the image obtained from vision acquisition was full of obstacles, such as a road sign,
a traffic cone and yellow box markings. All these were a hindrance to vehicle and object recognition
and matching. Noise reduction was achieved usingStop? an “image mask” to exclude irrelevant regions of
the image, “colour plane extraction” to convert the colour Yes
image to a binary image in only black and
white, and “basic morphology” to modify the shape of binary objects in the image, as well as to adjust
End
the brightness. After noise reduction, the indicator, traffic cone and yellow box markings had been
removed (blacked out) and onlyFigure the private car remained
3. Process flow of the on CVROSS.
the screen with its shape shown in white.

Figure
Figure4.4.An
Anexample of noise
example of noisereduction.
reduction.

3.2.2. Vehicle and

3.2.2. Vehicle and Object
Object Recognition
Recognition and
and Matching
Matching
Two
Two common
common matching
matching methods
methods are are provided
provided byby the
the CVROSS:
CVROSS: pattern
pattern matching and geometric
matching and geometric
matching.
matching. If all the items that need to be detected and matched share the same features, pattern
If all the items that need to be detected and matched share the same features, pattern
matching is the best method, as it will compare all the features and colours of an item from
matching is the best method, as it will compare all the features and colours of an item from the the template
template and the captured image. However, not all vehicles and objects are the same. For example,
some owners may paint the roof or body of a vehicle. Therefore, not all objects have the same patterns
or the same colours. This may negatively impact on the effectiveness of vehicle and object recognition
and matching. As a result, together with noise reduction (converting the captured image to a binary
image in only black and white), geometric matching seems more suitable for use in the CVROSS to
Sensors 2019, 19, 1796 9 of 26

and the captured image. However, not all vehicles and objects are the same. For example, some owners
may paint the roof or body of a vehicle. Therefore, not all objects have the same patterns or the
same colours. This may negatively impact on the effectiveness of vehicle and object recognition and
matching. As a result, together with noise reduction (converting the captured image to a binary image
Sensors
in only2019, 19,and
black x 9 of 25
white), geometric matching seems more suitable for use in the CVROSS to detect,
recognize and match different types of vehicles and objects based on their shapes, lengths and other
other significant features, as well as to determine the image score values mentioned. It can prevent
significant features, as well as to determine the image score values mentioned. It can prevent failure
failure of recognition and matching of an item due to different patterns and colours. In this case,
of recognition and matching of an item due to different patterns and colours. In this case, when the
when the image is acquired properly and noise has been reduced, the process of recognition and
image is acquired properly and noise has been reduced, the process of recognition and matching can
matching can then be carried out. Vehicle and object recognition and matching are based on templates
then be carried out. Vehicle and object recognition and matching are based on templates inserted
inserted during the set-up process for the system parameters. When an object appears, or a vehicle
during the set-up process for the system parameters. When an object appears, or a vehicle passes by or
passes by or parks inside the angle of view of the HD vision devices, the devices will capture images
parks inside the angle of view of the HD vision devices, the devices will capture images and compare
and compare them automatically with the templates in the database. Thus, vehicles and objects can
them automatically with the templates in the database. Thus, vehicles and objects can be assigned to a
be assigned to a category after recognition.
category after recognition.
3.3.Decision
3.3. DecisionSupport
Supportin
inRoadside
RoadsideParking
Parking

3.3.1.Evaluation
3.3.1. Evaluationof
ofParking
ParkingGaps
Gaps

InInthe the following

following sections,
sections, parking-gap
parking-gap calculations,
calculations, parking-space
parking-space evaluation evaluation
and decision and support
decision
support in parking are considered and evaluated, as shown
in parking are considered and evaluated, as shown in Figure 5. The block diagram shows that the entire in Figure 5. The block diagram shows
that the entire
computation computation
involves involves three
three components: components:
(i) conversion between (i) conversion
pixel valuesbetween and actual pixel values
scale for roadand
actual (ii)
traffic, scale fuzzyfor road
logic traffic, (ii) fuzzy
for vehicle parking logic for vehicle
reservation andparking reservation
(iii) decision support andfor (iii)parking
decision support
activities.
for parking
After vehiclesactivities.
and objects After
havevehicles and objects
been recognized andhave matched,been the recognized
CVROSS and matched,
calculates parking the CVROSS
gaps for
calculates
each individual parking gaps
traffic for each
lane. First,individual
the conversion trafficbetween
lane. First, thethe conversion
pixel betweenfrom
value collected the pixel
the imagevalue
collected from the image and the actual scale should be
and the actual scale should be implemented, via experimental studies. The actual scale of the road implemented, via experimental studies. The
actualisscale
traffic affectedof thebyroadthe traffic
heightisofaffected
the camera by the (Hheight
c ) fromofgroundthe camera level(H andc) from ground level
the viewing angleand of thethe
viewing
camera (θangle
c ). The of the
conversioncamera (θ
ratio c).
is The conversion
essential for ratio
estimating is essential
the actual for estimating
number of parkingthe actual
spaces number
and is
of parking spaces and is utilized in the following analysis. As
utilized in the following analysis. As indicated in Figure 6, in the first traffic lane, shown at the top indicated in Figure 6, in the first traffic
oflane,
the shown
image, at the top
there of thevehicles,
are three image, there and areeach three
of them vehicles,
has fourand each
corner of points,
them has i.e.,four {(x1corner
, y1 ), (xpoints,
2 , y2 ),
(xi.e.,
,
3 3y{(x), ,
1(x y 1,),y(x
4 4 )}2,⊆
y ),
2{X (x
1 , ,
3Y y
1 3),
} (x
for ,
4they 4)} ⊆
first {X 1, Y1} for
vehicle V 1 .the
In first vehicle
general, the V 1. In
four general,
corner the
points four
of corner
vehicle V points
n can
beof presented
vehicle Vn in canthe
beform
presented {(x4n-3in , ythe
4n-3form {(x,4n-3
), (x4n-2 , y4n-3
y4n-2 ), (x
), (x , ,yy4n-1
4n-2
4n-1 4n-2), (x4n-1,,y
4n y4n )}),⊆(x{X
4n-1 4n,ny, 4nY)}n },⊆where
{Xn, Ynn }, where
is the
n is the of
number numbervehicles of captured
vehicles captured
in the image. in the image.

Figure 5. Overview of the computations in the CVROSS.

Figure 5. Overview of the computations in the CVROSS.
Sensors 2019, 19, 1796 10 of 26
Figure 5. Overview of the computations in the CVROSS.

Figure
Figure6.6.Illustration
Illustrationofofcalculation
calculation of parking
parking gaps.
gaps.
Sensors 2019, 19, x 10 of 25
In this study, it is supposed that the cameras are mounted on street lights and nearby facilities,
so that theInheights anditviewing
this study, angles
is supposed thatofthe
thecameras
cameras aremay be different.
mounted on streetThus,
lightsadjustment of the images
and nearby facilities,
takensobythat
cameras that are
the heights andnot mounted
viewing on street
angles of the lights
cameras is needed, in order to
may be different. standardize
Thus, adjustmenttheofimage
the for
images taken
conversion. by cameras
Referring that are not mounted
to the government’s on street
street-light lights[34]
design is needed,
and other in order
workto[35,36],
standardize the
a conversion
ratio image for conversion.
mapping Referring to
can be established to deal
the government’s
with variousstreet-light
predefined design [34] and
heights and other
angleswork [35,36],
of the cameras,
a conversion
in order ratio mapping
to standardize canof
their field beview
established
(FOV).toFigure
deal with various predefined
7 illustrates the scenarioheights and anglescameras
of mounting of
the cameras, in order to standardize their field of view (FOV). Figure 7 illustrates the scenario of
at different heights with different viewing angles. In the default setting, the cameras that are mounted
mounting cameras at different heights with different viewing angles. In the default setting, the
on the street lights are set vertically, to cover a particular FOV. For other camera settings with different
cameras that are mounted on the street lights are set vertically, to cover a particular FOV. For other
heights andsettings
camera viewing angles,
with theheights
different imageand andviewing
the FOV are the
angles, then adjusted
image and theback
FOVto the
are default
then adjustedsetting.
To achieve
back to the default setting. To achieve the above goal, a conversion mapping profile is established via for
the above goal, a conversion mapping profile is established via experimental studies
several predefined
experimental configurations,
studies for several predefined the ratio La /Land
and thus configurations, p is thus
defined 𝐿 ⁄conversion
for the
the ratio 𝐿 is definedbetween
for
pixelthe
values and actual
conversion scales
between pixelfor roadand
values traffic.
actualConsequently,
scales for roadthe conversion
traffic. ratio the
Consequently, canconversion
be applied for
ratio can
calculating thebeactual
applied for calculating
length and width theofactual length and width of vehicles.
vehicles.

Other camera
Default (mounted
Y in street light)
setting (mounted
on facilities)

θ
Height of camera

Traffic Road
Field of View
X
Figure 7. Illustration of various camera positions.
Figure 7. Illustration of various camera positions.
After the four corner points of each vehicle have been retrieved, the CVROSS computes the
After the four corner points of each vehicle have been retrieved, the CVROSS computes the
maximum and minimum values of x and y, i.e., L(x,n) and L(y,n) , to represent the actual length and
maximum and minimum values of x and y, i.e., L(x, n) and L(y, n), to represent the actual length and
widthwidth
of vehicle VnVrespectively,
of vehicle according to Equations (1) and (2). Using the above information,
n respectively, according to Equations (1) and (2). Using the above information, the
the size of the vehicle captured by the cameraisisknown,
size of the vehicle captured by the camera known, and
and thethe spaces
spaces for for vehicle
vehicle parking
parking reservation
reservation
are then computed using fuzzy logic:
are then computed using fuzzy logic:
𝐿
L( , ) = [max(X ) − min(X )] ∙ La (1)
L(x,n) = [max(Xn ) − min(Xn )]𝐿 · (1)
Lp
𝐿
L( , ) = [max(Y ) − min(Y )] ∙ (2)
𝐿
In fuzzy logic, there are three processes: fuzzification, the inference engine and defuzzification.
These evaluate the reservation factor γ ∈ [0, 1] and the estimated time of stay ts from three inputs, i.e.,
parking time tp, L(x, n) and L(y, n). In fuzzification, the input and output parameters are fuzzified
Sensors 2019, 19, 1796 11 of 26

La
L( y,n) = [max(Yn ) − min(Yn )]· (2)
Lp
In fuzzy logic, there are three processes: fuzzification, the inference engine and defuzzification.
These evaluate the reservation factor γ ∈ [0, 1] and the estimated time of stay ts from three inputs, i.e.,
parking time tp , L(x,n) and L(y,n) . In fuzzification, the input and output parameters are fuzzified with a
set of defined fuzzy classes, such as “small”, “medium” and “high”, and the degree of belongingness
µ, all taking values between zero and one. For example, the parking time tp is fuzzified with its
corresponding membership functions to show the degree of belongingness, as in Equation (3) where
xi represents all elements in tp , µA (xi ) is the membership function of fuzzy class A in xi and n is the
total number of elements x. In the inference engine, the set of rules R = {R1 , R2 , R3 , . . . , Rm } is used
to evaluate the aggregated outputs from the input parameters, where m is the total number of rules
collected from interviewing the domain experts. The mechanism of the inference engine is referred to
as Mamdani’s method [37], where the consequences of the rules are expressed by fuzzy sets rather
than linear mathematical expressions. Equation (4) shows the inference process for obtaining the
aggregated outputs. The “OR” operator is applied for combining all the membership function values,
resulting in a bounded area in the output membership functions. In defuzzification, the outputs in
fuzzy sets are then converted back to crisp values x0 using the centroid method, which measures
the centre of gravity of the bounded area, as shown in Equation (5). In the application, when the
region of the vehicle is recognized in the image, the parking time, reservation factor and estimated
time of stay for the vehicle can be measured, to truly reflect the occupied spaces for vehicle parking.
The membership functions used in fuzzy logic are predefined intuitively by interviewing domain
experts and industrialists, as shown in Table 1:

n µA (xi )
X
tp = i=1 (3)
xi
n h  io
µB (Yi ) = max mini µA1 (x1 ), µA2 (x2 ), . . . , µA j x j (4)
R
0
µA (x)·xdx
x = R (5)
µA (x)dx

Table 1. Definitions of fuzzy classes and membership functions in the CVROSS.

Parameter/Unit Range Fuzzy Class Membership Function Type

Input:
Short [0, 5, 7] trimf 1
L(x,n) /m [0, 18] Medium [5, 7, 11, 12] trapmf 2
Long [11, 12, 18] trimf 1
Narrow [2, 2.5, 3] trimf 1
L(y,n) /m [2, 4] Medium [2.5, 3, 3.5] trimf 1
Wide [3, 3.5, 4] trimf 1
Night hour [0, 0, 6, 9] trapmf 2
tp /h [0, 24] Office hour [6, 9, 15, 18] trapmf 2
Evening hour [15, 18, 24, 24] trapmf 2
Output:
Slightly increased [0, 0.33, 0.5] trimf 1
γ [0, 1] Substantially increased [0.33, 0.5, 0.67] trimf 1
Significantly increased [0.5, 0.67, 1] trimf 1
Short [0, 60, 120] trimf 1
ts /min [0, 360] Medium [60, 120, 150, 210] trapmf 2
Long [150, 210, 360] trimf 1
Remarks: 1 trimf refers to the triangular shape of the membership functions; 2 trapmf refers to the trapezoid shape
of the membership functions.
Sensors 2019, 19, 1796 12 of 26

To calculate the parking gap between vehicles, four situations should be taken into consideration,
as shown in Figure 8. To prevent errors of unlimited value, the largest pixel value for length (the 640th
pixel rather than the first pixel), is utilized to compute the first gap, namely, the end gap (G0 ). Hence,
in accordance with Equation (6), the end gap (G0 ) can be computed by subtracting the maximum
x-value of the first vehicle V1 from the maximum pixel value for length, i.e., 640 pixels. For cases 1
and 2 in Figure 8, the calculation of the end gap is performed via Equation (6), and the end gap is the
partial parking gap between two vehicles, such that the information from the right camera should be
considered to measure the whole parking gap between the two vehicles (as for case 1 or 3). For cases 3
and 4 in Figure 8, since the maximum x-value of the first vehicle V1 is equal to the maximum pixel
value, this implies that the end gap is equal to zero and the length of the first vehicle captured by the
camera does not represent the actual length of the vehicle. The information from the right camera
should be collected and combined with the partial length of V1 to confirm the actual length of V1 ,
whilst the situation of the left camera should be similar to case 2 or 4. In the algorithm, the parking gap
Gn between vehicles n and n+1 can be generalized for one specific traffic lane, as shown in Equation (7).
The parking gap (in pixel values) can be obtained and can be converted back to the actual scale12using
Sensors 2019, 19, x of 25
the pixel-to-actual-scale conversion ratio. When considering cases 1 and 3, Xn+1 cannot be observed for
computing
right cameraGshould
n , so that
bemax(X n+1 ) and
collected is defined to bewith
combined zero,the
which defines
partial theof
length edge of the
V1 to captured
confirm image.
the actual
All the above situations were considered in the proposed system, and parking-space assignment
length of V1, whilst the situation of the left camera should be similar to case 2 or 4. In the algorithm, was
then
the conducted
parking gap G for three types
n between of vehicles:
vehicles a private
n and n+1 can becar, a cargo van
generalized forand
oneaspecific
truck, with regulatory
traffic lane, as
parking spaces of 5 m, 7 m and 11 m respectively [35].
shown in Equation (7). The parking gap (in pixel values) can be obtained and can be converted back
to the actual scale using the pixel-to-actual-scale conversion ratio. When considering cases 1 and 3,
Xn+1 cannot be observed for computingGG 0 n= Pixel
, so thatmax − max
max(X (X1 )defined to be zero, which defines the
n+1) is
(6)
edge of the captured image. All the above situations
X were
X considered Linp the proposed system, and
n−1 n
n = Pixelmax
parking-spaceGassignment was then(Xconducted
− max n+1 ) − for
i=0 G −
three
i j=1 [ L
types of(x,j ) ·γ j ·
vehicles: ],awhere
privaten car,
, 0 a cargo van
(7)
La
and a truck, with regulatory parking spaces of 5 m, 7 m and 11 m respectively [35].

Figure 8. Four
Figure possible
8. Four possiblecases
casesfor
for capturing images.
capturing images.

3.3.2. Parking Spaces and Decision Support Functionalities

𝐺 = Pixel − max(𝑋 ) (6)
Calculation of parking gaps is useful for computing the available parking spaces. After gathering
data on parking gaps in traffic lanes, the CVROSS can carry out further 𝐿 processing by dividing the
G = Pixel − max(X ) − 𝐺 − [𝐿( , ) ∙ 𝛾 ∙ ], where 𝑛 ≠ 0 (7)
length of each gap by the lengths of different types of vehicles ω (input
𝐿 as one of the parameters
described in Section 3.3.1), via Equation (8) for a truck, van and private car. If the length is less than
the length of a private car, the objects are classified as “others”, e.g., obstacles in the road. Therefore,
3.3.2. Parking Spaces and Decision Support Functionalities
Calculation of parking gaps is useful for computing the available parking spaces. After gathering
data on parking gaps in traffic lanes, the CVROSS can carry out further processing by dividing the
length of each gap by the lengths of different types of vehicles ω (input as one of the parameters
described in Section 3.3.1), via Equation (8) for a truck, van and private car. If the length is less than
Sensors 2019, 19, 1796 13 of 26

the information on available parking spaces (Struck , Svan and Sprivate car ) for the three types of vehicles
in each individual traffic lane, or even for the whole road, is produced, to inform road users about
real-time roadside occupancy and vacancy. Consequently, the proposed system can provide three
decision support functionalities: evaluation of average space utilization, measurement of loading and
unloading activity and average waiting time for parking. Considering that there are p cameras in the
whole traffic lane, the average space utilization (U) is calculated by dividing the total available parking
gaps by the maximum length of the image (in pixels), as shown in Equation (9), where Gij represents
the available parking gap i determined by camera j and Pixel(j,max) represents the maximum pixel value
of camera j. If the traffic space is occupied by trucks and cargo vans instead of private cars, these are
regarded as engaging in loading and unloading activities. The indication of performing loading and
unloading activities (LUA) is calculated by dividing the sum of the lengths of the vehicles, for L(x,n) ≥ 5,
by the actual total length of the traffic lane (Ltraffic ) under surveillance by camera p, as in Equation (10).
The indication of loading and unloading activities is assumed to be updated hourly in the proposed
system, to conveniently track the traffic situation. For average waiting times for parking, the proposed
system will determine the waiting time for the entire traffic lane when any available parking space Gn
is less than the required parking space for a private car (representing the smallest parking space for the
three types of vehicles). Therefore, users can make an appropriate decision according to the above
three indicators:
1 La
Sk = ·[Gn · ], where k = truck, van or private car; ω = 5, 7 or 11 (8)
ω Lp
Pp Pn
j=1 1=1 Gij
U = Pp (9)
j=1
j·Pixel( j,max)
Pp Pn j
j=1 i=1 L(x,i) j
LUA = , for L(x,i) ≥ 5 (10)
Ltraffic
In addition, the CVROSS was run with a timestamp control. Frequently updating real-time
information allows road users to obtain useful information about real-time roadside occupancy and
vacancy. Thus, road users can make good use of the information to make real-time decisions, such as
parking their vehicles on the road or finding other roads. In addition, running the program with a time
delay can prevent overrunning and overloading of the server, thus maintaining a high level of stability
and accuracy of calculation.

3.4. Evaluation of the Proposed System

The evaluation of the proposed CVROSS system is twofold: (i) validation of parking-gap
estimations and (ii) system performance from the perspectives of drivers and property management
companies. The parking-gap estimation is validated using a paired sample t-test for examining the
difference between two sets of 50 sample data points, (i.e., estimated and actual parking gaps). This is
aimed at investigating the appropriateness of parking-gap estimations in the CVROSS. On the other
hand, the satisfaction and the system performance are evaluated by interviewing drivers and property
management company representatives, using a survey. Figure 9 shows sample questions used to
obtain feedback. The survey was conducted in two timeframes: (i) before implementing the CVROSS
and (ii) three months after implementing the CVROSS. As a result, a comparative summary before and
after implementing CVROSS was produced, for further statistical analysis.
Sensors 2019,19,
Sensors2019, 19,1796
x 14
14of
of26
25

Figure 9. Sample of survey questions for evaluating system performance.

Figure 9. Sample of survey questions for evaluating system performance.
4. Case Study
4.1. Site Selection
To validate the feasibility and performance of the proposed system, a case study was conducted,
In this phase,
implementing data collection
the CVROSS in Kwunwas Tongmainly focused
District, on siteDue
Hong Kong. visits in Kwun
to the Tongand
seriousness District, Hong
urgency of
Kong, in order to gather traffic information from real situations for further analysis.
the problem of traffic congestion in Hong Kong, particularly in Kwun Tong District, an IoT-based system The detailed site
visits
for were carried
surveillance out on Shing
of roadside loading Yipand
Street and Hing
unloading YipisStreet,
bays much next to Hung
needed. To Road
The entire and a branch
implementation
of Hoi Yuen Road in Kwun Tong District, which are the busiest roads in the area (as
was divided into three phases: (i) site selection, (ii) deployment of the CVROSS and (iii) establishmentshown in Figure
10).
of The data user
web-based collected included
interfaces. Thetraffic facilities
project and information
commenced on the surroundings
with data collection mainly fromofselected
the selected
site
roads, such as the number of lamp posts, traffic lights and traffic lanes, together with
visits focusing on Kwun Tong District. Having obtained a better understanding of traffic situations and distances and
the lengths and widths of the roads. Data on traffic situations in the area considered
occupation, a simulation model was built based on traffic features and real cases in Kwun Tong District. were collected
through observation,
Subsequently, a solutiontowith
obtain
the aCVROSS
better understanding of thewas
system architecture realdeployed
state of to
occupation
tackle the and
issuetraffic
with
congestion.
the help of a web application. Thus, implementing the CVROSS involved the application of computer
vision, cloud computing, big data analysis and reusable energy (solar power), to detect, recognize and
match vehicles and objects, hence providing road users with comprehensive and real-time information,
after image processing. The information was also visualized using a front-end web interface to
enhance understandability.

4.1. Site Selection

In this phase, data collection was mainly focused on site visits in Kwun Tong District, Hong Kong,
in order to gather traffic information from real situations for further analysis. The detailed site visits
were carried out on Shing Yip Street and Hing Yip Street, next to Hung To Road and a branch of Hoi
Yuen Road in Kwun Tong District, which are the busiest roads in the area (as shown in Figure 10).
The data collected included traffic facilities and information on the surroundings of the selected roads,
such as the number of lamp posts, traffic lights and traffic lanes, together with distances and the lengths
and widths of the roads. Data on traffic situations in the area considered were collected through
observation, to obtain a better understanding of the real state of occupation and traffic congestion.
Figure 10. Selected traffic lanes for site visits.

As a major industrial area, Kwun Tong District sees a large number of loading and unloading
activities every day. Firstly, vehicles temporarily double-parked for loading or unloading or waiting
for roadside spaces to become available, are common in the area of interest. Because a large number
of trucks usually double-park on Hing Yip Street, serious traffic congestion can occur. Secondly, as
parking spaces are scarce in Kwun Tong District, some nearby companies might occupy the roadside
with objects such as traffic cones and boards, to preserve parking spaces. This situation hinders other
road users from using the road. Therefore, in addition to detection, recognition and classification of
of Hoi Yuen Road in Kwun Tong District, which are the busiest roads in the area (as shown in Figure
10). The data collected included traffic facilities and information on the surroundings of the selected
roads, such as the number of lamp posts, traffic lights and traffic lanes, together with distances and
the lengths and widths of the roads. Data on traffic situations in the area considered were collected
through
Sensors 2019,observation,
19, 1796 to obtain a better understanding of the real state of occupation and 15traffic
of 26
congestion.

Figure
Figure10.
10.Selected
Selected traffic lanesfor
traffic lanes forsite
site visits.
visits.

As a major
major industrial area, Kwun Tong Tong District
District sees
sees aa large
large number
number of of loading
loading and
and unloading
unloading
activities every day.
day. Firstly,
Firstly, vehicles
vehicles temporarily
temporarily double-parked
double-parked for for loading
loading or
or unloading or waiting
for roadside spaces to become available, are common in the area area ofof interest.
interest. Because a large number
of trucks usually
usually double-park
double-parkon onHing
HingYip
YipStreet,
Street,serious
serioustraffic
trafficcongestion
congestion cancan occur.
occur. Secondly,
Secondly, as
as parking
parking spaces
spaces areare scarce
scarce inin KwunTong
Kwun TongDistrict,
District,some
somenearby
nearbycompanies
companiesmightmight occupy
occupy the roadside
with
with objects
objects such
such as
as traffic
traffic cones
cones and
and boards,
boards, to
to preserve
preserve parking
parking spaces.
spaces. This situation hinders other
road users from using
using the
the road.
road. Therefore, in addition to detection, recognition and classification of
various vehicles and objects, the computer vision-based roadside surveillance system needs to provide
road users and logistics companies with information about occupancy and vacancy, so that they can
optimize fleet schedules based on analytical information via self-regulation.
External walls of buildings and lamp posts are the only possible positions for installing the
HD vision devices for capturing images in the computer vision-based surveillance system. There
is some difficulty in installing HD vision devices on the external walls of buildings, particularly on
commercial buildings, without permission. It is believed that most property owners are likely to refuse
to install the HD vision devices due to a lack of benefits and effects on the appearance of their buildings.
Furthermore, various buildings may have different features at different heights, and this may lead
to difficulties in unifying standards, such as the height of all HD vision devices, thus negatively
affecting vision and possibly creating some overlaps. Therefore, lamp posts, managed by the Highways
Department of the Hong Kong Government, are recommended as the best places to install the HD
vision devices along roads and streets. In the areas considered, lamp posts have a mounting height of
10 m, set by the Highways Department [32]. The Highways Department is responsible for preventive
and corrective maintenance of lamp posts. This is beneficial for the installation and operation of
the vision devices and computer vision-based surveillance system, as breakdowns and errors can be
resolved promptly to maintain a high level of stability in the system, compared with installations on
the external walls of buildings.

4.2. Deployment of the CVROSS

In the design of the CVROSS, the deployment of the proposed system consists of four major
components: (i) noise reduction, (ii) vehicle recognition, (iii) calculation of parking gaps and (iv)
calculation of available parking spaces. The proposed system was deployed using Simulink and
LabVIEW for algorithm modelling and real-world prototyping respectively, as shown in Figure 11.
The models and algorithms for parking-gap calculation and parking-gap assignment and the fuzzy
logic for vehicle parking reservation, were developed in the Simulink environment, while the user
 In the design of the CVROSS, the deployment of the proposed system consists of four major
components: (i) noise reduction, (ii) vehicle recognition, (iii) calculation of parking gaps and (iv)
calculation of available parking spaces. The proposed system was deployed using Simulink and
LabVIEW for algorithm modelling and real-world prototyping respectively, as shown in Figure 11.
Sensors 2019, 19, 1796 16 of 26
The models and algorithms for parking-gap calculation and parking-gap assignment and the fuzzy
logic for vehicle parking reservation, were developed in the Simulink environment, while the user
interface and the system prototyping and data acquisition elements, were controlled and constructed
in the LabVIEW environment.

Figure 11.11.
Figure System
Systemdeployment environmentofof
deployment environment the
the CVROSS.
CVROSS.

4.2.1. Noise Reduction

4.2.1. Noise Reduction
Firstly,
Firstly,the
theset-up
set-upof ofthe
theparameters
parametersfor forreal-life
real-lifeimplementation
implementation was
wasrequired,
required, especially thethe
especially size of
size
vehicles and traffic cones and their dimensions in pixels, according to Section 3.2.
of vehicles and traffic cones and their dimensions in pixels, according to Section 3.2. Then, data Then, data collection
commenced via vision acquisition
collection commenced and noise reduction.
via vision acquisition and noise As shown inAsFigure
reduction. shown 12,inthe original
Figure image
12, the from
original
vision
image acquisition
from visionincluded
acquisitiontheincluded
traffic-lane
thelines, which lines,
traffic-lane are unrelated
which are tounrelated
vehicle and object recognition
to vehicle and object
and matching.
recognition and These lines are likely to negatively influence the results of recognition and matching.
Sensors 2019, 19, x matching. These lines are likely to negatively influence the results of recognition and
16 of 25
It was found
matching. that,
It was afterthat,
found noiseafter
reduction, only the only
noise reduction, shapestheof related
shapes of vehicles remained.
related vehicles This could
remained. This
facilitate the subsequent
could facilitate processing
the subsequent of the image.
processing of theAs a result,
image. As better and better
a result, more accurate
and more processing
accurate
could
processing could be achieved, to enhance the effectiveness of the designed system. roads
be achieved, to enhance the effectiveness of the designed system. However, different with
However,
their ownroads
different featureswithand characteristics
their own features may
andrequire differentmay
characteristics techniques
requirefor noise reduction,
different techniquestoforremove
noise
unrelated signals from an image. This may represent a time-consuming
reduction, to remove unrelated signals from an image. This may represent a time-consuming modification when the system
is applied to different
modification when theroadssystemin Hong Kong.
is applied to different roads in Hong Kong.

Figure 12.12.
Figure Image
Imagedifferences
differences before andafter
before and afternoise
noise reduction.
reduction.

4.2.2. Vehicle Recognition

To determine the dimensions of specific vehicles, the technique of geometric matching was used
to recognize and match two trucks with different appearances. It was found that the CVROSS was
able to detect, recognize and match vehicles from all traffic lanes on the road and those in an
individual traffic lane. General traffic conditions could be interpreted using the information from the
matching results for all traffic lanes on the road, for example, for the issue of vehicles remaining
double-parked. On the other hand, information from an individual traffic lane was capable of
Sensors 2019, 19, 1796 Figure 12. Image differences before and after noise reduction. 17 of 26

4.2.2. Vehicle Recognition

4.2.2. Vehicle Recognition
To determine the dimensions of specific vehicles, the technique of geometric matching was used
To determine
to recognize the dimensions
and match two trucks of specific vehicles,
with different the technique
appearances. It of
wasgeometric
found thatmatching was used
the CVROSS wasto
recognize and match two trucks with different appearances. It was found that
able to detect, recognize and match vehicles from all traffic lanes on the road and those in an the CVROSS was able to
detect, recognize
individual trafficand
lane.match vehicles
General trafficfrom all traffic
conditions lanesbeon
could the road and
interpreted those
using theininformation
an individual traffic
from the
lane. General traffic conditions could be interpreted using the information from
matching results for all traffic lanes on the road, for example, for the issue of vehicles remainingthe matching results for
all traffic lanes on the road, for example, for the issue of vehicles remaining
double-parked. On the other hand, information from an individual traffic lane was capable of double-parked. On the other
hand, information
illustrating frominana individual
situations particular traffic
traffic lane
lane,was
to capable
determine of illustrating
the level ofsituations in a particular
traffic congestion and
traffic lane, to determine the level of traffic congestion and identify loading
identify loading and unloading activities. Figure 13 illustrates the use of geometric matching in and unloading activities.
Figure
vehicle13 illustrates the
recognition. Theuse of geometric
coordinates matching
of vertices areinlocated
vehicleforrecognition.
measuringThe thecoordinates of vertices
corresponding length
are
andlocated
width of forthe
measuring
vehiclestheviacorresponding length and width of the vehicles via a geometric technique.
a geometric technique.

Figure 13.13.
Figure AnAn
illustration
illustrationof
ofgeometric matchingbyby
geometric matching thethe CVROSS.
CVROSS.

4.2.3. Calculation of Parking Gaps

Using the matching results, the calculation of parking gaps was implemented. The roadside
situations (at most six gaps, including the end gap) could be determined by one HD video camera.
The total number of gaps was set to a large number M for the implementation. By applying the
CVROSS, the parking gaps (considering various types of vehicles) could be identified and computed,
for cases where Lp , Hc and θc are well-defined. Figure 14 shows the fuzzy logic toolbox running in
the MATLAB runtime compiler on the server side of the CVROSS. The input and output—with their
corresponding membership functions as given in Table 1—were created, and the fuzzification process,
Mamdani’s inference and the defuzzification process were defined. Thus, the fuzzy capability of
parking reservation can be included in the computation of parking gaps.
 CVROSS,
for cases where Lp, Hc and
the parking gapsθc(considering various
are well-defined. types
Figure 14ofshows
vehicles) could logic
the fuzzy be identified
toolbox and computed,
running in the
for cases where
MATLAB runtime Lp, compiler
Hc and θc onare the
well-defined.
server sideFigure
of the14CVROSS.
shows the fuzzy
The logic
input andtoolbox
outputrunning
– with in the
their
MATLAB runtime
corresponding compiler functions
membership on the server side of
as given in the CVROSS.
Table Thecreated,
1 – were input and andoutput – with their
the fuzzification
corresponding
process, Mamdani’s membership
inference functions
and theasdefuzzification
given in Tableprocess1 – were created,
were and Thus,
defined. the fuzzification
the fuzzy
process,
capability Mamdani’s
Sensors 2019,of
19,parking
1796 inference and the defuzzification process were defined.
reservation can be included in the computation of parking gaps. Thus, the 18
fuzzy
of 26
capability of parking reservation can be included in the computation of parking gaps.

Figure 14. Fuzzy logic in MATLAB runtime compiler.

Figure
Figure 14.14.
Fuzzy
Fuzzylogic
logicin
in MATLAB
MATLAB runtime runtime compiler.
compiler.
Figure 15 illustrates the lengths of the parking gaps in three lanes with different numbers of
Figure
vehicles. When15 illustrates
there wasthe no lengths
vehicle of in the
laneparking
1, the endgapsgap in three
was lanes with as
displayed different numbers
640 pixels, which of
vehicles.
vehicles. When
indicated When there
that all therewas
the nono
was
pixelsvehicle inavailable,
vehicle
were lane 1, the1,
in lane endthegap
while thewas
end displayed
gap
otherwasgaps as 640zero
displayed
had pixels,
as 640 which
value. indicated
pixels,
From which
the
that all
indicated the pixels
that were
all the available,
pixels while
were the other
available, gaps
while had
the zero value.
other
computations, it was found that the algorithm was able to calculate the parking gaps gapsFrom
had the computations,
zero value. From it was
the
found that the algorithm
computations,
programmatically, it based
was found
onwastheable
thatto the
previous calculate the results.
algorithm
matching parking
was ablegaps
For programmatically,
to calculate
example, based
the parking
if no vehicles ongaps
are in the
the
previous
traffic lane,matching
programmatically, results.
only the based
end onFor
gap isthe example,
previous
shown. As aifmatching
no vehicles
result, are in
results.
information Forthe
on traffic
example,
the lengthlane,
ifofnoonly
all the end
vehicles
parking are
gapsgap
incan is
the
shown.
traffic
be produced As a
lane, onlyresult,
and theused information
end
forgap on
is shown.
further the length
As a result,
processing. of
However,all parking
information gaps
the numberon the can be
of length produced and
of all parking
gaps needed used
to be setgaps
beforefor
can
further
be produced
running processing.
and used
the program. However,
Thus, the number
for further
further of gaps
processing.
calculation was needed
However, to be
the
required set before
number
regarding running
of gaps
the the program.
needed
capacity to
of be Thus,
set before
a particular
further
road andcalculation
running the
traffic lane.was
program. required
Thus,
Then, the regarding
further
possible the capacity
calculation
number was
of ofgaps
required
parking a particular
that mayroad
regarding the
need and traffic
capacity
to be of alane.
computed Then,
particular
was
the possible number of parking gaps that may need to be computed was
road and traffic lane. Then, the possible number of parking gaps that may need to be computed was
set. set.
set.

Figure
Figure 15. 15. Parking
Parking gapsininthe
gaps thethree
three individual
individual traffic lanes.
traffic lanes.
Figure 15. Parking gaps in the three individual traffic lanes.
4.2.4. Calculation of Available Parking Spaces
The calculation of available parking spaces was then tested by applying the CVROSS,
after obtaining the data on parking gaps. Based on the proposed algorithm, the length of each
parking gap (in pixels) was divided by the length of each type of vehicle and the constant reserved for
parking that particular vehicle. The number of vehicles available for parking in the individual traffic
lane was computed by the algorithm. If the number of private cars (which are the shortest vehicles in
the scenario) was equal to zero, this meant that a particular parking gap was wasted. Furthermore,
the CVROSS was able to add up the lengths of all vehicles and available parking spaces, and show
these as occupancy and vacancy respectively, as well as showing the wasted spaces in the individual
traffic lane, to provide users with information on the general traffic conditions. Therefore, by dividing
the length of parking gaps by the length of each type of vehicle, the CVROSS—using the function
for calculation of available parking spaces—proved its ability to compute, and provide users with,
information about the number of spaces available for parking in each area. Thus, it is useful and
CVROSS was able to add up the lengths of all vehicles and available parking spaces, and show these
as occupancy and vacancy respectively, as well as showing the wasted spaces in the individual traffic
lane, to provide users with information on the general traffic conditions. Therefore, by dividing the
length of parking gaps by the length of each type of vehicle, the CVROSS – using the function for
calculation of available parking spaces – proved its ability to compute, and provide users 19with,
Sensors 2019, 19, 1796 of 26
information about the number of spaces available for parking in each area. Thus, it is useful and
helpful to users in making immediate decisions and for self-regulation. For example, if there are no
helpfulany
longer to users in making
available parking immediate decisions
spaces, drivers and
can for self-regulation.
decide For example,
to park their vehicles if there
on other are no
roads, to
longer any available parking spaces, drivers can decide to park their vehicles
prevent traffic congestion occurring. Furthermore, the CVROSS is also applicable to objecton other roads, to prevent
traffic congestion
recognition occurring.
and matching. Furthermore,
Figure 16 showsthe
theCVROSS is also
aggregation of applicable to objectspaces
available parking recognition and
in specific
matching.
lanes. The Figure
data on16parking
shows the gapsaggregation of available
collected from several parking
camerasspaces in specific lanes.
are summarized to form The thedata
set on
of
parking gaps collected from several cameras are summarized to form the set of aggregated
aggregated results of parking gaps and used to assign various types of vehicles to the empty gaps. results of
parking
Thus, gaps
the and used
CVROSS to assign
is still able tovarious types
compute theofparking
vehicles gaps
to theand
empty gaps. Thus,
available parkingthespaces
CVROSS to is still
show
able to compute the parking gaps and available parking spaces to show occupancy
occupancy and vacancy, as well as the wasted spaces, for the situation where there is occupation by and vacancy, as
well
an as the
object inwasted spaces,
the traffic lane.for the situation where there is occupation by an object in the traffic lane.

Figure
Figure 16. Calculation
16. Calculation of of availableparking
available parking spaces
spaces in
inthe
thecase
caseofof
occupation by an
occupation byobject.
an object.
4.3. Establishment of Web-Based User Interface
4.3. Establishment of Web-Based User Interface
It is crucial to provide users with a good-quality interface for information visualization that
It is crucial to provide users with a good-quality interface for information visualization that is
is easy to understand and simple to use. The CVROSS is able to offer users various forms of
easy to understand and simple to use. The CVROSS is able to offer users various forms of information
information visualization. The results of vehicles detected and matched, as well as the parking
visualization. The results of vehicles detected and matched, as well as the parking gaps and available
gaps and available parking spaces computed, can be presented graphically. An interface design for
information visualization is presented in Figure 17. An image display was utilized to present real-time
traffic situations on the road within the system time. This could provide users with a general view
of the areas under surveillance. In addition, as shown in Figure 18, on the main dashboard of the
CVROSS, various reports about road usage by different types of vehicles can be generated for road
users, logistics companies and the public, for better understanding of traffic situations in the areas
under surveillance in a particular period.
visualization is presented in Figure 17. An image display was utilized to present real-time traffic
situations on the road within the system time. This could provide users with a general view of the
areas under surveillance. In addition, as shown in Figure 18, on the main dashboard of the CVROSS,
various reports about road usage by different types of vehicles can be generated for road users,
logistics
Sensors companies
2019, 19, 1796 and the public, for better understanding of traffic situations in the areas 20
under
of 26
surveillance in a particular period.

Figure 17. 17.

Figure Overview ofofthe
Overview theinterface
interface design forinformation
design for information visualization.
visualization.

Figure
Figure18.
18.Main
Maindashboard ofthe
dashboard of theCVROSS.
CVROSS.

5. Results
5. Results andand Discussion
Discussion
Following the
Following the case
case study,
study, it
it was
was found
found that
that it
it was
was feasible
feasible to
to implement
implement thethe proposed
proposed system
system in
in
real-life situations, to provide functionalities for real-time monitoring and decision support
real-life situations, to provide functionalities for real-time monitoring and decision support for roadside
for
parking
roadsideactivities. On the one
parking activities. Onhand, property
the one companies
hand, property can evaluate
companies the severity
can evaluate of double-parking
the severity of double-
and view the real-time roadside situation from the back office. During the case
parking and view the real-time roadside situation from the back office. During the case study, 100 samples
study, 100
of parking gaps were examined for accuracy of parking gap estimation, compared with the actual
measured parking gap. The accuracy comparison between estimated and actual parking gaps is shown
in Figure 19. A paired sample t-test was applied to examine the difference between estimated and
actual parking gaps, and the null hypothesis was to assume that the mean difference was zero. It was
found that statistical significance for the mean difference was achieved with a p-value of 0.02; thus,
the measurement of parking gaps using the CVROSS can be considered feasible. The average and
samples
samples of of parking
parking gaps gaps were
were examined
examined for for accuracy
accuracy of of parking
parking gap gap estimation,
estimation, compared
compared with with the
the
actual
actual measured parking gap. The accuracy comparison between estimated and actual parking gaps
measured parking gap. The accuracy comparison between estimated and actual parking gaps
is
is shown
shown in in Figure
Figure 19.19. A A paired
paired sample
sample t-test
t-test waswas applied
applied to to examine
examine the the difference
difference between
between
estimated
estimated
Sensors
and
and
2019, 19,
actual parking gaps, and the null hypothesis was to assume
actual parking gaps, and the null hypothesis was to assume that the mean difference
1796
that the mean difference
21 of 26
was
was zero.
zero. ItIt was
was found
found that
that statistical
statistical significance
significance for for the
the mean
mean difference
difference was was achieved
achieved withwith aa p-p-
value
value ofof 0.02;
0.02; thus,
thus, the
the measurement
measurement of of parking
parking gapsgaps using
using the
the CVROSS
CVROSS can can bebe considered
considered feasible.
feasible.
maximum
The
The average
average errors
and for parking
and maximum
maximum gap estimation
errors
errors for parkingwere
for parking gap 1.47 m andwere
gap estimation
estimation 3.60 1.47
were m, respectively.
1.47 m
m andand 3.60
3.60 m,
m,Moreover,
respectively.the
respectively.
accuracy
Moreover,
Moreover, theof estimation
the accuracy
accuracy ofof the time of
of estimation stay
estimation of was
of the also
the time
time ofassessed
of stay using
stay was
was also 50 samples
also assessed to
assessed using compare
using 50 estimated
50 samples
samples to to
and actual
compare times
estimated of stay,
and as shown
actual in
times Figure
of stay,20.asA paired
shown sample
in Figure t-test
20. A
compare estimated and actual times of stay, as shown in Figure 20. A paired sample t-test was alsowas also
paired applied
sample to examine
t-test was the
also
difference
applied
applied to tobetween
examineestimated
examine the and actual
the difference
difference between
between times of stay, and
estimated
estimated the null
and actual
actual hypothesis
times
times of stay,was
of stay, andto
and the
theassume
null the mean
null hypothesis
hypothesis
difference
was
was to was
to assume
assume the zero. It
the mean was found
mean difference that
difference was statistical
was zero.
zero. It significance
It was
was found
found that for the mean
that statistical difference
statistical significance
significance forwas achieved
for the
the mean
mean
with a p-value
difference was of 0.015;
achieved thus,
with the
a estimation
p-value of of time
0.015; of
thus, stay
the can be
estimationconsidered
of time
difference was achieved with a p-value of 0.015; thus, the estimation of time of stay can be considered feasible.
of stay canThebe average and
considered
maximum
feasible. errors in the estimation of time of stay were 1.70 h and 4.64
feasible. The average and maximum errors in the estimation of time of stay were 1.70 h and 4.64 h,
The average and maximum errors in the estimation of time of h, respectively.
stay were 1.70 On
h the
and other
4.64 h,
hand, truckersOn
respectively.
respectively. Onandthedrivers
the othercan
other make
hand,
hand, use of the
truckers
truckers andproposed
and drivers
drivers cansystem
can make
make to understand
use
use of
of the the specificsystem
the proposed
proposed roadside
system to
to
situation.
understand
understand In the
the next
the section,
specific
specific the performance
roadside
roadside situation.
situation. In Inofthe
CVROSS
the next is assessed
next section,
section, theby
the conducting of
performance
performance a comparative
of CVROSS
CVROSS is is
analysis—before
assessed by and
conducting after
a adopting
comparative the proposed
analysis – system.
before and Any
after advantages
assessed by conducting a comparative analysis – before and after adopting the proposed system. Any
adopting the and contributions
proposed system. are
Any
discussed
advantages
advantages accordingly.
and
and contributions
contributions are are discussed
discussed accordingly.
accordingly.

Figure
Figure 19. Accuracy
19.19.
Figure Accuracycomparison
Accuracy comparison for
comparison parking-gap
for parking-gap
for parking-gap measurement.
measurement.
measurement.

Figure 20.
Figure
Figure Accuracy
20.20. Accuracycomparison
Accuracy comparison for
comparison for estimation
estimationofof
for estimation time
time
of of of
time of stay.
stay.
stay.
5.1. Comparative Analysis of the CVROSS
To verify the performance of the proposed system, a comparison of before and after the use of the
CVROSS was made, considering three aspects: (i) severity of traffic congestion, (ii) energy savings
Sensors 2019, 19, 1796 22 of 26

of vehicles and (iii) driver satisfaction. The results were obtained by interviewing 50 drivers and
10 representatives of property management companies in the selected areas. These were selected
because they were frequent users of Shing Yip Street and Hing Yip Street and had considerable
management responsibility. Table 2 shows the findings from the interviews with property management
company representatives and individual drivers. In summary, the effects obtained by implementing the
proposed system appear to be positive. According to property management companies, the severity
of traffic congestion on specific roads and incidences of double-parking were reduced by 41.2% and
33% respectively. Moreover, companies can save on the labour force costs of controlling the busy
roadside situation, reducing numbers of workers from 10 per shift to six per shift. Monitoring and
control of roadside situations can be conducted in the back office, and real-time traffic information can
be provided to truckers and drivers via the proposed system. Drivers and truckers commonly agreed
that average fuel consumption was reduced, and that the average time to locate suitable parking
spaces could be decreased by 51.6%, as they were able to go to specific parking areas after receiving
information from the proposed system. In addition, truckers and drivers were generally satisfied
with the proposed system, as it could improve the poor situation regarding double-parking and traffic
congestion in busy districts.

Table 2. Comparative analysis before and after using the CVROSS.

Before Using After Using % of

No. Area UoM a
CVROSS CVROSS Improvement
Perspectives from property management companies
1 Severity of traffic congestion Scale (1–10) b 8.5 5.0 −41.2%
2 Severity of double parking Scale (1–10) b 9.7 6.5 −33.0%
Labour force on controlling
3 people per shift 10 6 −40.0%
roadside activities
Perspectives from drivers and truckers
1 Average fuel saving Scale (1–10) b 7.8 5.1 −34.6%
Average time to locate suitable
2 min 18.2 8.8 −51.6%
parking space
3 Average driver satisfaction Scale (1–10) b 6.1 8.2 +34.4%
Notes: a UoM refers to unit of measurement; b Scale (1–10) refers to a Likert scale from 1 to 10, while 10 is the highest
score in scale and 1 is the lowest score in scale.

5.2. Timestamp Control of the CVROSS

A while loop (a control flow statement that allows code to be executed repeatedly) can also be
utilized in the servers. This allows the program to be run every second or even more frequently,
also enabling data, information and reports to be saved every second or more frequently. Although this
can provide users with regularly updated information, the server overloads easily, as large quantities
of data, information and reports must be processed and saved. Thus, the stability of the system is
negatively affected. For example, assuming a month has 30 days, there are 2,592,000 s in a month
(for an Excel file generated programmatically utilizing a while loop). This means that there are
2,592,000 records per month, generating approximately 31,536,000 records per year. This could be
problematic for big data analysis after a number of years, as more data are generated.
As a result, a timed loop is suggested instead of a while loop. In this case, the timed loop is set with
a five-minute delay. Thus, it can still update the real-time traffic information frequently for road users,
facilitating the operations of the CVROSS and loading the server smoothly, to prevent overloading and
to maintain stability, facilitating the process of big data analysis. In fact, the five-minute delay can be
adjusted, based on the real needs of road users, hence providing them with a more user-friendly system.
Sensors 2019, 19, 1796 23 of 26

5.3. Significance of the CVROSS

The feasibility and the performance of the CVROSS have been demonstrated by the case study
and implementation. The proposed system makes three major contributions to research and society:
(i) smart parking for roadside operations, (ii) applied artificial intelligence for roadside parking
activities and (iii) an environmentally-friendly business model for property management companies.

5.3.1. Smart Parking for Roadside Operations

In the field of urban development, smart cities are thought to be a future trend and emerging
technologies are applied to formulate different forms of decision support and intelligence, to improve
efficiency and effectiveness. In the evolution of the smart city, smart mobility is specific to objects
(including human beings), transportation and logistics. The ontology of smart parking is developed
from smart transportation, which is an active research area. In this paper, smart parking for roadside
operations was applied, to eliminate double-parking and enhance roadside occupancy. Via the adoption
of IoT technologies, the new topic of smart parking has been explored to address the problem of
double-parking at the roadside. Therefore, novel contributions relating to smart parking have been
made in this paper.

5.3.2. Applied Artificial Intelligence for Roadside Parking Activities

In the evaluation of parking gaps and available parking spaces, the proposed system makes use
of fuzzy logic to classify various types of vehicles in real-life situations, i.e., private cars, cargo vans,
trucks and obstacles. Fuzzy logic offers flexibility and intelligence in the algorithm, to generate certain
decision support functionalities. Therefore, the proposed system is able to evaluate reservation spaces
and the estimated time of stay of the vehicles. This information can be used to estimate average
space utilization, loading and unloading activity and average waiting times for parking. Overall,
the data collected by IoT technologies and the data on roadside activities are integrated using artificial
intelligence techniques, (i.e., fuzzy logic), to create value in industrial applications.

5.3.3. Green Business Model Using the IoT

Considering the findings from implementing the proposed system, it can be concluded that the
proposed system has advantages with respect to energy-saving, time efficiency and better roadside
occupancy. For example, if there are always many trucks loading and unloading on Monday mornings,
some logistics companies can plan to change their schedules in order to load or unload goods at
other times, to prevent waiting at times of traffic congestion. Thus, the system is able to facilitate
traffic and fleet management by self-regulation. The interface and related information can be further
amended and transferred to a mobile application to enhance the transparency of roadside activities.
Via self-regulation by road users and logistics companies (taking advantage of information and
communication technologies), the system relieves traffic congestion, achieves an efficient road network
and facilitates the development and management of a reliable and intelligent transport system.
This work is not only beneficial to property management companies and drivers, but also has a positive
influence on Hong Kong society, fostering an environmentally friendly and safe atmosphere in roadside
operations. By adopting the CVROSS, companies could save on costs and labour power for managing
roadside activities, and thus business profitability could be improved.

6. Conclusions
Roadside activities, such as loading and unloading, negatively affect traffic situations if not
kept under control. Hence, smart mobility is crucial for built-up areas in Hong Kong, aligning with
smart-city development. This is especially the case where no real-time information about the roadside
activities, occupancy and vacancy is accessible to the general public and road users. The CVROSS is a
fully integrated solution, equipped with a set of wireless HD vision devices enabling image capture
Sensors 2019, 19, 1796 24 of 26

from the roadside, utilizing machine learning and supported by solar power, proposed as a real-time
IoT-based system for surveillance of roadside loading and unloading bays. This can facilitate traffic
and fleet management by implementing smart mobility, thus achieving a highly efficient road network.
This paper contributes to the utilization of IoT technologies by developing the CVROSS in the Simulink
and LabVIEW environments. From the set-up of parameters to vehicle and object recognition and
matching with noise reduction, the calculation of parking gaps and available parking spaces and,
lastly, to information visualization, the CVROSS, together with state-of-the-art IoT technologies, is able
to provide road users with real-time roadside traffic information, such as roadside occupancy and
vacancy, thereby enhancing the transparency of roadside activities. Various reports, e.g., daily reports,
can also be produced for different parties via big data analysis. For example, logistics companies
can optimize fleet schedules based on analytical information. In addition, fuzzy logic is applied in
evaluating parking gaps and available parking spaces at the roadside, to establish decision support in
roadside operations for enhancing evaluation accuracy and system flexibility. This is expected to help
alleviate traffic congestion by reducing waiting times for loading and unloading activities and reducing
costs of fuel and energy consumption by locating parking vacancies and preventing circling around
the roads. The proposed CVROSS solution facilitates the development and management of a reliable
and intelligent transport system in Hong Kong, resulting in the achievement of smart parking on the
basis of smart mobility and smart transportation. Future efforts can be made to investigate, modify
and realize the implementation of the CVROSS, to enhance the transparency of roadside activities.
Ultimately, through the application of an IoT-based surveillance system for roadside loading and
unloading bays, the development and management of a reliable and intelligent transport system in
Hong Kong can be facilitated. In future work, the proposed system can be implemented in other
regions affected by double-parking and busy roadside activities. In addition, automated and intelligent
methods for formulating membership functions and fuzzy rules may be proposed.

Author Contributions: Conceptualization, G.T.S.H., C.H.W. and W.H.W.; Data curation, Y.P.T. and C.H.W.;
Project administration, G.T.S.H. and C.H.W.; Software, G.T.S.H., Y.P.T. and C.H.W.; Supervision, C.H.W. and
K.L.C.; Writing—original draft, G.T.S.H., Y.P.T. and C.H.W.; All authors discussed the results and commented on
the manuscript.
Funding: The authors would also like to thank the Research Office and the Department of Industrial and Systems
Engineering (ISE), The Hong Kong Polytechnic University (PolyU) for their financial assistance to the involved
research student (Project Code: RUDV), and the Department of Supply Chain and Information Management
(SCM), The Hang Seng University of Hong Kong (HSUHK) for financial assistance to the involved student helpers.
Our gratitude is extended to the Big Data and Artificial Intelligence Group of HSUHK, and ISE of PolyU, for their
support in this work. Special thanks for all involved student helpers (SCM, HSUHK) and technical assistants
(ISE, PolyU).
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Cerchecci, M.; Luti, F.; Mecocci, A.; Parrino, S.; Peruzzi, G.; Pozzebon, A. A low power IoT sensor node
architecture for waste management within smart cities context. Sensors 2018, 18, 1282. [CrossRef]
2. Inglis, R.P.S.; Brenner, R.P.; Puzo, E.L.; Walker, T.O.; Anderson, C.R.; Thomas, R.W.; Martin, R.K. A secure
wireless network for roadside surveillance using radio tomographic imaging. In Proceedings of the 2012
6th International Conference on Signal Processing and Communication Systems, Gold Coast, Australia,
12–14 December 2012.
3. Samuel, A.L. Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 2000, 44,
206–226. [CrossRef]
4. Lyon, D. Surveillance Studies: An Overview; Polity Press: Cambridge, UK, 2007; pp. 13–18.
5. Valera, M.; Velastin, S.A. Intelligent distributed surveillance systems: A review. IEE P-Vis. Image Sign. 2005,
152, 192–204. [CrossRef]
6. Figueiredo, L.; Jesus, I.; Machado, J.T.; Ferreira, J.R.; De Carvalho, J.M. Towards the development of intelligent
transportation systems. In Proceedings of the ITSC 2001. 2001 IEEE Intelligent Transportation Systems,
Oakland, CA, USA, 25–29 August 2001.
Sensors 2019, 19, 1796 25 of 26

7. Problems in the Urban Environment: Traffic Congestion and Its Effects. Available online: https://ro.uow.edu.
au/wollgeo/14/ (accessed on 1 March 2019).
8. Qin, M.; Wang, Y.; Cui, Z.; Zhu, Y. The design and realization of an advanced urban traffic surveillance and
management system. In Proceedings of the 2006 6th World Congress on Intelligent Control and Automation,
Dalian, China, 21–23 June 2006.
9. Gualotuña, T.; Macías, E.; Suárez, Á.; Rivadeneira, A. Low Cost Efficient Deliverying Video Surveillance
Service to Moving Guard for Smart Home. Sensors 2018, 18, 745. [CrossRef] [PubMed]
10. Maha Vishnu, V.C.; Rajalakshmi, M. Road side video surveillance in traffic scenes using map-reduce
framework for accident analysis. Biomed. Res. 2016, 27, 257–266.
11. Huang, L. Roadside camera calibration and its application in length-based vehicle classification.
In Proceedings of the 2010 2nd International Asia Conference on Informatics in Control, Automation
and Robotics (CAR 2010), Wuhan, China, 6–7 March 2010.
12. Coifman, B.; Beymer, D.; McLauchlan, P.; Malik, J. A real-time computer vision system for vehicle tracking
and traffic surveillance. Transport. Res. C-Emer. 1998, 6, 271–288. [CrossRef]
13. Saran, K.B.; Sreelekha, G. Traffic video surveillance: Vehicle detection and classification. In Proceedings
of the 2015 International Conference on Control Communication & Computing India (ICCC), Trivandrum,
India, 19–21 November 2015.
14. Wu, Q.; Zhang, Y. Parking Lots Space Detection. Available online: https://pdfs.semanticscholar.org/569c/
55651e2f978fe9b0801799aa02cc587a2640.pdf (accessed on 1 January 2019).
15. Lin, S.F.; Chen, Y.Y.; Liu, S.C. A vision-based parking lot management system. In Proceedings of the 2006
IEEE International Conference on Systems, Man and Cybernetics, Taipei, Taiwan, 8–11 October 2006.
16. Micheloni, C.; Foresti, G.L.; Snidaro, L. A cooperative multicamera system for video-surveillance of parking
lots. In Proceedings of the IEE Symposium on Intelligent Distributed Surveillance Systems, London, UK,
23–26 February 2003.
17. Wilson, J.; Patwari, N. Radio tomographic imaging with wireless networks. IEEE T Mobile Comput. 2010, 9,
621–632. [CrossRef]
18. Martin, R.K.; Anderson, C.; Thomas, R.W.; King, A.S. Modelling and analysis of radio tomography.
In Proceedings of the 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor
Adaptive Processing, San Juan, Puerto Rico, 12–15 December 2011.
19. Anderson, C.R.; Martin, R.K.; Walker, T.O.; Thomas, R.W. Radio tomography for roadside surveillance.
IEEE J. Sel. Topics Signal Process 2014, 8, 66–79. [CrossRef]
20. Wohwe Sambo, D.; Yenke, B.O.; Förster, A.; Dayang, P. Optimized Clustering Algorithms for Large Wireless
Sensor Networks: A Review. Sensors 2019, 19, 322. [CrossRef] [PubMed]
21. Cheung, S.Y.; Ergen, S.C.; Varaiya, P. Traffic surveillance with wireless magnetic sensors. In Proceedings of
the 12th ITS world congress, San Francisco, CA, USA, 6–10 November 2005.
22. Traffic Surveillance by Wireless Sensor Networks: Final Report. Available online: http://citeseerx.ist.psu.edu/
viewdoc/download?doi=10.1.1.84.6488&rep=rep1&type=pdf (accessed on 2 March 2019).
23. Tiwari, P.; Saxena, V.P.; Mishra, R.G.; Bhavsar, D. Wireless sensor networks: Introduction, advantages,
applications and research challenges. Int. J. Technol. Innovations Res. 2015, 14, 1–11.
24. Buch, N.; Velastin, S.A.; Orwell, J. A review of computer vision techniques for the analysis of urban traffic.
IEEE Trans. Intell. Transp. Syst. 2011, 12, 920–939. [CrossRef]
25. Computer Vision: Evolution and Promise. Available online: http://cds.cern.ch/record/400313/files/p21.pdf
(accessed on 2 March 2019).
26. Perallos, A.; Hernandez-Jayo, U.; Zuazola, I.J.G.; Onieva, E. Intelligent Transport Systems: Technologies and
Applications; John Wiley & Sons: New York, NY, USA, 2015.
27. Beymer, D.; McLauchlan, P.; Coifman, B.; Malik, J. A real-time computer vision system for measuring traffic
parameters. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern
Recognition, San Juan, Puerto Rico, 17–19 June 1997.
28. Huang, M.C.; Yen, S.H. A real-time and color-based computer vision for traffic monitoring system.
In Proceedings of the 2004 IEEE International Conference on Multimedia and Expo, Taipei, Taiwan,
27–30 June 2004.
29. Neethu, N.J.; Anoop, B.K. Role of Computer Vision in Automatic Inspection Systems. Int. J. Comp. App.
2015, 123, 28–31.
Sensors 2019, 19, 1796 26 of 26

30. Zander, S.; Nguyen, T.; Armitage, G. Automated traffic classification and application identification using
machine learning. In Proceedings of the IEEE Conference on Local Computer Networks 30th Anniversary,
Sydney, Australia, 17 November 2005.
31. Rosten, E.; Porter, R.; Drummond, T. Faster and better: A machine learning approach to corner detection.
IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 105–119. [CrossRef] [PubMed]
32. Hong Kong Planning Standards and Guidelines. Available online: https://www.pland.gov.hk/pland_en/
tech_doc/hkpsg/sum/pdf/sum.pdf (accessed on 2 March 2019).
33. Brailean, J.C.; Kleihorst, R.P.; Efstratiadis, S.; Katsaggelos, A.K.; Lagendijk, R.L. Noise reduction filters for
dynamic image sequences: A review. Proc. IEEE 1995, 83, 1272–1292. [CrossRef]
34. Public Lighting Design Manual Third Edition. Available online: https://www.hyd.gov.hk/en/publications_
and_publicity/publications/technical_document/public_lighting_design_manual/doc/PLDM_2016.pdf
(accessed on 2 March 2019).
35. Lin, C.-C.; Wang, M.-S. A Vision Based Top-View Transformation Model for a Vehicle Parking Assistant.
Sensors 2012, 12, 4431–4446. [CrossRef] [PubMed]
36. Chen, L.W.; Ho, Y.F. Centimeter-Grade Metropolitan Positioning for Lane-Level Intelligent Transportation
Systems Based on the Internet of Vehicles. IEEE T. Ind. Inform. 2019, 15, 1474–1485. [CrossRef]
37. Hamzah, A.; Shurman, M.; Al-Jarrah, O.; Taqieddin, E. Energy-Efficient Fuzzy-Logic-Based Clustering
Technique for Hierarchical Routing Protocols in Wireless Sensor Networks. Sensors 2019, 19, 561. [CrossRef]
[PubMed]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).