b_28.jpg
 
  • Guidelines
  • Recommendations
  • Search
  • Background information
  • About Proceed
 
Switch Language to english Switch Language to german Switch Language to france Switch Language to polish
Background topics
Methods for Market analysis
   Fields of interest with regard to basic analysis
   Approach on market analysis in The Netherlands
   The quality loop
   Self-assessment models
   Measurement of customers satisfaction and performance
   Benchmarking
   Standardisation and certification
   Quality partnerships
   Service guarantee and service charters
Network and Infrastructure
Financing
Management
Marketing
 
Download Guidelines (pdf):
English
German
French
Polish
Background Information

This background information can be used for educational purposes, and for obtaining more in-depth knowledge.



Monitoring of performance

Public Transport Authorities can measure customers’ satisfaction and performance by standardised measurement methods like the Customer Satisfaction Survey. Operators can make use of self-assessment models to assess their own production quality.
Explanation:

Performance measurement data provides public transport management with objective assessments of current supply with regard to service quality. Regularly monitored aspects regarding the existing service include, for instance, punctuality of bus services, likelihood of getting a seat, cleanliness of buses, and condition of bus stops. Key management uses of a performance measurement system include: Service monitoring; Evaluation of financial performance; Management functions; Internal communications; Development of service design standards; Communication of achievements and challenges; and Promotion of community benefits.

The quality loop approach has been accepted as an EN standard but it is not the only option. Under the standardised measurement procedure described in the European Standard EN 13816:2002, three relevant methods for service quality measurement are described:

  • Direct Performance Measurements (DPM)
  • Performance measurement is the process whereby an organization establishes the parameters within which programmes, investments, and acquisitions are measured for desired results. This process of measuring performance often requires the use of statistical evidence to determine progress toward specific defined organizational objectives.

  • Mystery Shopping Surveys (MSS)
  • Mystery shopping is a tool used by market research companies to measure quality of retail service or gather specific information about products and services. Mystery shoppers posing as normal customers perform specific tasks—such as purchasing a product, asking questions, registering complaints or behaving in a certain way – and then provide detailed reports or feedback about their experiences.

  • Customer Satisfaction Surveys (CSS).
  • One can measure customers’ satisfaction by using CSS. The definition of the threshold value for a certain quality criterion depends on the community’s service quality level. This can vary considerably across Europe, but the measurement method can be used universally. The methods above are standardised procedures (see Annex C of EN 13816:2002; see examples from EN15140:2006).

Figure 1-2: Linkage between different tools for quality management in public transport

(PORTAL 2003)

Regarding production quality, the operator can make use of self-assessment. A self-assessment model known as EFQM model (The European Foundation for Quality Management “Excellence model”) is widely used in other industries and is also suitable for the public transport industry.

A benchmark is a standard of excellence of achievement against which other similar things, services or products are measured or judged. Standardisation and certification are part of the quality assurance process. Quality partnerships are schemes drawn up by a local authority aimed at implementing a local bus strategy where responsibilities are set between different actors.

Tools discussed in connection with quality management and continuous improvement processes in public transport (Figure 1‑2) are explained in more detail as ► Background information: The quality loop / Self-assessment models / Benchmarking / Standardisation and certification / Quality partnership / Service guarantees and service charters.

Critical issues:

If a certain quality level is agreed in a contract, the actual quality needs to be checked regularly by independent standardised procedures during the whole contract period. This applies especially to cities with gross contracts (where the revenue risk is taken by the public transport authority), since the motivation of the operator to provide the intended quality is not given by the market itself.

Surveys in which the judgements of surveyors or customers are involved cannot exclude personal or emotional judgements. The contributions of such effects to determining essential financial consequences like bonuses or penalties for contract partners therefore have to be considered carefully .

Punctuality of buses is not always within the scope of the operator’s management. Heavy traffic may interfere with public transport punctuality. On the other hand the reliability of service is strongly connected to maintenance (bus depots), dispatching, and availability of spare resources (in vehicles and staff) etc. In small cities, where no computer-based operation control system is in use, the monitoring of punctuality requires expensive measurement procedures.

Good practice examples:
  • Brighton & Hove (UK): Customer service is a fundamental ethos of the bus operator, and Brighton & Hove buses have one of the highest customer satisfaction rating in the UK. Monitoring of performance is a key element in achieving high customer satisfaction, and the performance of service operation (including performance of front-line staff) is monitored on a regular basis. There is a draft Bus Punctuality Improvement Plan that represents a joint commitment (between the city authority and the bus operator) to achieving continuous improvement in the punctuality and reliability of bus services for an agreed period of 5 years, extending thereafter by mutual consent. The data from the city’s Real-Time Passenger Information Systems is a fundamental element in this.
  • Hasselt, Leuven, Sint-Niklaas (Belgium): The Flemish public transport operator ‘De Lijn’ has a policy of continuously measuring customer satisfaction by written surveys on the bus. Every 3 months 11,000 interviews are performed which are then combined with the results of a 2-year household survey concerning satisfaction measurements. In both measurements 11 quality factors are surveyed: Drivers’ attitude, punctuality, information and communication, interconnections, comfort (e.g. seating likelihood) and capacity, safety, cleaning, frequency and regularity, or price.
  • Groningen (The Netherlands): A Bonus / penalty – also based on punctuality - is used in Groningen; derived from customer feedback.
  • Larissa (Greece): A timetable reliability programme is used in Larissa city, which exhibits high frequencies and good reliability. More specifically, a daily-performance monitoring programme is implemented by bus drivers and bus owners. With this programme potential deviations from schedule are reported and evaluated.
  • Luzern (Switzerland): The bus operator is ISO 9001:2000 and ISO 14001 certified. These international standards regarding quality and environment management ensure that customers receive a service that has a high quality with high safety, and is environmentally friendly. Furthermore, the operator works according to the OHSAS-standards 18001:1999 that considers Health and Safety at work. Also, quality reviews from Switzerland Tourism and other awards have been obtained. Customer satisfaction is measured every second year, and the questions regarding the public transport service refer to such items as supply (availability), cleanliness of buses and bus stops, and friendliness of the drivers, etc. The ÖVL and the Cantonal Department of Public Transport initiated this survey, and in 2004 the public transport service achieved 71 out of 100 points in the rating of customer satisfaction.
  • Ljubljana (Slovenia): The international standard EN 13816:2002 is implemented as a Mystery Shopping Survey to perform quality control. The cleanliness of vehicles is measured; but the public transport authority uses the results only for information.
  • Luleå (Sweden): The 'condition' of the bus stops (cleanliness, illumination, damage etc.) is regularly monitored through a customer satisfaction index.
  • Kaunas, Klaipeda (Lithuania): Every autumn trolleybus and bus operators are evaluated by a customer satisfaction index that is calculated by monitoring passengers’ opinion of the service quality provided, including aspects such as punctuality, cleanliness of vehicles, driving culture, politeness of drivers etc.
    In Kaunas city the city authorities regularly monitor punctuality, cleanliness and the state of the bus stops. An authority control team monitors the operators’ work without giving any prior notice. If any major deviations are detected penalties are imposed on the operators.
    In Klaipeda there is a website for public transport, where regular monitoring of customers’ satisfaction with regard to the public transport service quality takes place. The city’s public transport authorities regularly monitor punctuality, cleanliness and the state of the bus stops. Also, public transport time-schedule punctuality is checked by a vehicle traffic control system where data is sent using GPRS.
  • Rijeka (Croatia): In Rijeka internal control of punctuality is performed regularly, based on data from the GPS-based system.
  • Zürich (Switzerland): Zurich public transport operator (VBZ) has two quality standards regarding a vehicle’s passenger capacity: a) The average annual passenger load of a vehicle run should not exceed 2 pers./m2 standing area. b) The run with the highest passenger load should not exceed an annual average of 4 pers./m2 standing area. The database is derived from the on-board automatic passenger counting system.
References and background reading:

EN 13816:2002 Public passenger transport - Service quality definition, targeting and measurement.

EN 15140:2006 Public passenger transport - Basic requirements and recommendations for systems that measure delivered service quality.

EQUIP (2000) The EQUIP Handbook for the self-assessment of the performance of local public transport services. Self-published by EQUIP Consortium. Download: http://www.transport-research.info/Upload/Documents/200310/equip.pdf

PORTAL Consortium (2003) Benchmarking & Quality Management in public transport. Written material 2003. Self-published by PORTAL Consortium. Download: http://www.eu-portal.net/material/downloadarea/kt1a_wm_en.pdf

Related guidelines:

Monitoring the performance of operation

Background information
Background information: The quality loop

The concept of the service quality loop (Figure 1‑3) distinguishes between:

  • Customer view, and
  • Service provider view.

Where:

  • "Service providers" are: Operators, Authorities and also Police and Road (Infrastructure) departments (these can be seen also as the "production" side)
  • "Customers" are users and also the community

Figure 1-3: The quality loop (PORTAL 2003)

The service provider view distinguishes between "targeted" service qualities and "delivered" service quality. Targeted service quality is set by authorities. There could be a difference between tendered quality and actually contracted quality; which implies a quality partnership as an agreement.

The service quality definition is a "set of quality criteria and appropriate measures for which the service provider is responsible". Level of quality is the "sum of weighted quality criteria".

The service quality targeted (by the provider) should correspond to the quality sought (by customers). The discrepancies between sought and targeted quality are influenced mostly by budgetary and technical constraints.

This results in a set of targeted quality criteria values, written down "in numbers", which are mostly thresholds.

The customer view differentiates between perceived quality (by users) and expected quality (by community or users associations).

Background information: Self-assessment models

The EFQM self-assessment model: For several years, the EFQM has been developing an efficient model of self-assessment for quality management at the level of a company or of a production system. The EFQM defines self-assessment as “taking a hard look at your organisation and scoring it against an ideal or model (the EFQM model in this case). The results indicate the organisation’s strengths and areas for improvement and provide the basis for future strategy and improvement plans…”. In the public transport sector, self-assessment can certainly lead to an improved knowledge of system and company performances.

The EQUIP model: EQUIP was an EU funded research project under the 4th Framework Programme which developed a self-assessment method for public transport operators, as a first step in a benchmarking process. The approach is described in a handbook (EQUIP 2000).

Background information: Measurement of customers’ satisfaction and performance

The measurement can be used for a large number of criteria, which have an impact on the quality of a public transport system. In some cases an “in conformity” approach is used:

  • Service quality (e.g. friendliness of drivers)

  • Vehicle quality (e.g. cleanliness)

  • Driving quality (additionally, customer feedback can be used for monitoring purposes).
  • Punctuality of services: The punctuality of service is “in conformity” if 95% of departures are on time. A departure is judged as “on-time” if the delay is less than three minutes and the departure is not more than one minute early. The measurement method is DPM. Quality control for punctuality can also be monitored by real-time information systems.

  • Bus stop quality (e.g. cleanliness, and accuracy of stop information).

  • Information tools (e.g. availability): The availability of service is “in conformity” if 99% of phone calls are answered, if 95% of calls are answered within three minutes and if 90% of answers are correct. The measurement method in this case is DPM by Mystery Shoppers.

  • Sales facilities (e.g. availability): The quality of ticketing service is “in conformity” if 90% of customers succeed in buying the most appropriate ticket on the ticket vending machine. The measurement method in this case is MSS.

Background information: Benchmarking

A Benchmark is a standard of excellence or achievement against which other similar production units or organisations are measured or judged. The basic idea of benchmarking is for the organisation to locate ‘competitors’ level of performance in a certain field that is superior to its own, and by doing that some item that is worthy of emulation is identified. Benchmarking is a process which - simply speaking - consists of the organisation:

  • Figuring out what to benchmark,

  • Finding out what the benchmark is (what is the standard of excellence?),

  • Determining how it is achieved (What methods or processes produce those results?),

  • Deciding to make changes to its own business practices that will enable it to meet or even to exceed the benchmark.

Best Practice is the means by which this ‘benchmark’ level of performance is achieved. Benchmarking can be described as the systematic comparison of the performance of an organisation against that of:

  • Other departments/subsidiaries (internal benchmarking)
    Internal benchmarking is not specific to public transport. Administrative, financial or other general management practices are benchmarked between departments inside many companies using value analysis techniques.

  • Other organisations or competitors (external benchmarking)
    External benchmarking between operators is not very common. The main reasons are confidentiality, the lack of efficient tools to identify comparable practices, and a lack of “no blame” cultures, often combined with a reticence to openness.

The main goal of benchmarking is to build on the successful experiences of others instead of "re-inventing the wheel".

Background information: Standardisation and certification

Standardisation and certification are part of the quality assurance process. Quality assurance consists of “all the planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfil given requirements for quality”. The standard defines the “systematic activities” and certification; the assurance that the standard will be respected.

The International Organisation for Standardisation (ISO) defines standards as “documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose”. AFNOR, the French organisation for standardisation, published a French standard (ref. NF X50-805) entitled “Quality within transportation services – Identification of the quality criteria for passenger transport” in April 1997, and has revised and improved it since.

Background information: Quality partnerships

The concept of “quality partnership” is relatively recent in the public transport sector. A quality partnership is a scheme drawn up by a local authority aimed at implementing a local bus strategy, in which both the authority and signatory operators commit to certain Actions. It first appeared in the UK in the beginning of the 1990s as a consequence of the deregulation and privatisation in 1986 of the UK Bus Industry (outside London and Northern Ireland). Prior to deregulation local authorities had had a formal service co-ordination role, although the benefits that this achieved varied between areas. Quality partnerships emerged under deregulation from a recognition that good partnership working was essential to growing the business for bus travel. The concept is generally only applicable to cases where bus services are run commercially outside any contract with a public authority, since in the case of a contracted operation the contracting authority can specify the level, type and quality of service to be delivered.

Background information: Service guarantee and service charters

The urban public transport user has a certain level of expectation and is not concerned with the way the service provider manages production activities in order to reach this level. The user is concerned with the service (Does the service fulfil my expectation?) and one of the main expectations concerns the reliability of the service. Consequently, the question is: How can the urban public transport user “trust” the service?

The concept of the service guarantee has been developed as an answer to this question. An operator or responsible authority may wish to be able to provide the urban public transport user with a guarantee of service level or quality.. Ideally, the guarantee should be applicable at every hour of the day, every season of the year, and anywhere on the network.

© 2007 PROCEED All rights reserved. | Legal Disclaimer