:

The Denpasar city government uses website facilities to display information on the procurement of goods and services. Publication of information through the website is intended to create a transparent, effective, efficient and accountable procurement system. From it ’ s initial use since 2013, this system has never been tested to determine whether the system is still in good condition. This study aims to evaluate the system based on user loyalty using the PIECES method. So that the website manager gets input about the quality of the website from the user's point of view. The results of the tests that have been carried out in this study indicate that the Performance Indicators scored 4.2 in the Very Satisfied category, the Information Indicators obtained a value of 4.2 in the Very Satisfied category, Economic Indicators obtained a value of 4.1 in the Satisfied category, Control and Security Indicators obtained a value of 4.3 in the Very Satisfied category, the Efficiency indicator got a score of 4.3 in the Very Satisfied category, the Service indicator got a score of 4.2 in the Very Satisfied category.


INTRODUCTION
The development of information distribution is now unstoppable because information can be obtained freely via the internet [1]. The use of information technology has become an absolute requirement for a government or private institution in order to provide excellent service to the community or customer [2] [3]. The services obtained by the community and customers are the rights that must be fulfilled by the institution [4]. With an information system, data processing becomes easier [5]. The use of structured information technology can reduce the budget for labor and equipment costs [6], besides that it can affect services to the community [7]. One of the media that is often used as a trusted source of information is a website. One of the media as a trusted source of information is the website. Website is a collection of pages that can be used to display text, images, animation, sound, and video [8]. Websites can be used to reach audiences far from business centers, online education systems, and for data publication in government [9] [10]. Many government and private agencies use the website as a medium to convey information or services. The more information that is displayed through the website, it will have an impact on the performance of the website. The worse the website performance, the more visits it will have [11]. Visitor satisfaction is an impression obtained when using a system without finding significant obstacles [12].
The City Government of Denpasar in particular uses the website as a medium to create a more transparent, effective, efficient and accountable goods and service procurement system. The Denpasar City Government Procurement of Goods and Services website can be accessed via eproc.denpasarkota.go.id, the existence of this website began in 2013. The website displays information about the procurement of goods and services owned by the Denpasar City government within a certain period of time. The Denpasar City Government has used an electronic-based procurement system for almost less than 10 years to cut business processes in the process of procuring goods and services which were originally manual [13]. In addition, the purpose of having an electronic-based procurement system is to invite the public to an active in the development process in Denpasar City so that the procurement process becomes transparent, effective, efficient and accountable. In addition, internal parties from the Denpasar City Government as parties who have work will reduce funds for the preparation of the procurement process because all needs have been provided on the website system. Individuals or private parties from all over Indonesia who have registered with the system can view and bid on the procurement of goods and services displayed on the website. As time goes by, there will definitely be more and more data stored on the website, so that it can make the system performance not good, it is necessary to periodically check website performance so that system visitors feel comfortable when accessing website pages. From the beginning of use until now there has never been an evaluation from the website manager about the quality of the website. Seeing from this anxiety, it is necessary to evaluate the system based on system user loyalty.
One method that can be used to evaluate the system is the pieces method. The PIECES method is a method that can be used to measure the performance of an information system [14]. For problem solving, the PIECES method uses several categories or components [15] [16]. The components of the assessment using the PIECES method include performance, information, economy, control and security, efficiency, and service [17]. This method was chosen because several components in it represent the aims and objectives to be achieved from this research, where the PIECES method is used to identify problems that exist in the system. The other methods have fewer components that are not as complete as the measurement components that are owned by the PIECES method. The output generated from testing the system with the PIECES method is the value of the level of system user satisfaction based on the assessment component. These results will provide an overview to system managers regarding the 6 (six) components that form the basis for measuring system performance, so that system quality can be maintained.
This study aims to determine the performance of the Denpasar City Goods and Services Procurement Website using the PIECES method, and the output of the test results will be able to provide an overview of website performance results and input to website managers about the components that can maintain website performance so that it can be maintained properly. so that the process of procuring goods and services can run according to the expectations of stakeholders in the City Government of Denpasar.

METHOD
The research method uses a quantitative descriptive method to describe a condition against several others as a benchmark for conducting an evaluation. Descriptive research is research that is conducted to determine the variables or components of a study without making comparisons or associating them with other variables. Based on the time of research completion, this research is included in the type of cross-sectional research where data processing, analysis, and conclusion drawing are carried out in one period [18]. The object of research is the Denpasar City Government Goods and Services Procurement Website, which is located at https://eproc.denpasarkota.go.id. The research stages carried out to complete this research include the following:

A. DATA COLLECTION
Data collection is a series of systematic procedures aimed at obtaining research data. Some data collection techniques include observation, interviews, and literature studies [19]. Data collection in this study goes through several stages, including: 1. A Literature study one of the steps in research for examine something that has been done before [20]. At this stage, researchers explore knowledge to find out about the PIECES method, things that need to be done to measure system user satisfaction, and similar research that is used as a tool to solve research problems. Some similar studies provide information about variables that can be used to make measurements, stages of research completion, and examples of results from research. 2. In field studies, researchers made direct observations at the Denpasar City Regional Secretariat Goods and Services Procurement office to obtain the necessary data. Researchers also conducted an interview process with system administrators who used the system directly. 3. Distribution of questionnaires is one of the tools used to collect data in this study to get an overview of visitor satisfaction with the system that has been running. The questionnaire consists of 22 questions asked to respondents [21]. Questionnaires were distributed to 100 system users consisting of internal and external parties of the Denpasar city government who were registered in the Denpasar city government goods and service procurement system. Internal parties are job owners who will seek external parties to complete the work, external parties are individuals or companies that will compete to bid for jobs displayed on the website. The questionnaire was used in this study because according to the author it is the best method that can be used to get responses from users who use the system directly.

B. PARTICIPANTS
To gain system user loyalty, questionnaires were distributed containing questions according to the components described in Table 2. The number of questions in this research questionnaire were 22 questions, the list of questions is in accordance with table 3. Questionnaires were distributed to active and registered system users.
The object tested in this study is the Denpasar City Goods and Services Procurement website. Participants consisted of internal and external users of the system from the Denpasar city government. Internal system users are officials who have jobs that will be displayed on the website, while external system users are providers who use the system to participate in job auctions that are displayed on the website.
The number of respondents who participated in this study amounted to 100 people. the number of respondents in detail can be seen in table 1. Table 1 Respondent

Number of Respondents Percentage
External Users 49 49 Internal Users 51 51

C. ASSESSMENT INDICATORS
The assessment indicators that are used as a reference in measuring website performance are listed in table 2. Based on Table 2, it can be explained that the performance assessment component is intended to measure system performance, which includes system output, response time, page completeness, consistency, and system tolerance to client requests. Furthermore, while the information component of the measurement is devoted to the accuracy, relevance, presentation, and flexibility of the information produced by the system, the economic component measures the reusability and system resources used and generated. The control security component measures the integrity and security of the system owned by the system for the convenience of system users in maintaining the privacy of visitor data. The efficiency component measures the usability and maintainability of the system, which is aimed at how reliable the system is in overcoming the problems faced by the client, and finally, the services component measures the accuracy, reliability, and simplicity of the system. There are many programs that can be reused in other applications

Resource
The development of the system requires a few resources Control and Security

Integrity
The system has limited access rights to operators for certain programs Security Data security in the system Efficiency Usability User effort when studying and operating the system (ease of operation) Maintainability The user's effort in overcoming the error Services Accurate Exactly or not the work process carried out by the system Reliability Trust the performance of the system as desired.

Simplicity
User understanding of the ease of use of the system D.

DATA ANALYSIS AND EVALUATION
The data analysis stage is used to process the results of the questionnaire data that has been distributed. The resultant data that has been obtained is then classified using the Likert scale. The LIKERT scale used is expressed in abbreviations ranging from STB, or very bad, to SB, or very good [22]. To determine the level of system satisfaction, a formula according to the LIKERT method is used, namely: The average satisfaction value obtained is then converted into a satisfaction predicate using Kaplan and Norton's theories [23] [24]. The value conversion to the level of satisfaction is as shown in table 5. Quite Satisfied 3,40 -4, 19 Satisfied 4,20 -5,00 Very Satisfied

E. RESULT
The research results are the final stage of the research. After data analysis, the results of the research were analyzed using the PIECES Method. This method also provides conclusions and suggestions for research. Conclusions and suggestions will be followed by an evaluation of the system, which PIECES components need to be improved.

RESULTS AND DISCUSSION A. VALIDITY TEST RESULTS
The validity test is used to ensure that the questions in table 3 have a validity value so that the questionnaire is feasible to be distributed to respondents. The results of testing the validity of the questionnaire can be seen in table 6.

B. RELIABILITY TEST RESULTS
Reliability test in this study the authors used the Alpha Cronbach formula. The author's tool to use to avoid human error is SPSS software version 20.0. The benchmarks for interpreting the degree of validity and reliability of the instrument obtained are in accordance with Table 7. Table 7 Reliability Interpretation Coefficient Criteria 0,00 < r ≤ 0,21 Very Not Good 0,21 < r ≤ 0,40 Not Good 0,61 < r ≤ 0,80 Pretty Good 0,61 < r ≤ 0,80 Good 0,81 < r ≤ 1,00 Very Good The results of the reliability test obtained a value of 0.968021398 with a total of 22 question items. Based on table 7, the reliability value produced is included in the very good criteria, so that the statement components and answers from the questionnaire can be declared reliable so that further data processing can be carried out.

Performance
The test results on performance indicators are poured into Table 9, and then data processing is carried out using Formula (1), so that the following results are obtained: Based on Formula (1), the test results in Table 9 obtained the following calculation JSK (total number of overall scores) = 2112, JK (total number of respondents' answers) = 500.
So that the value of RK is 2112 divided by 500 is 4.224 if the rounded up the result is 4.2, and based on table 5 the results are obtained with the very satisfied category.

Information
The test results on performance indicators are poured into Table 10, and then data processing is carried out using Formula (1), so that the following results are obtained: Based on Formula (1), the test results in table 10 obtained the following calculation JSK (total number of overall scores) is 2076 and JK (total number of respondents' answers) is 500.
So that the value of RK is 2076 divided by 500 is 4.152 if rounded up the result is 4.2, and based on Table 4 the results are obtained in the very satisfied category.

Economics
The test results on performance indicators are poured into Table 11, and then data processing is carried out using Formula (1), so that the following results are obtained: Based on Formula (1), the test results in Table 10 obtained the following calculation JSK (total number of overall scores) is 1239 and JK (total number of respondents' answers) is 300.
So that the value of RK is 1239 divided by 300 is 4.13 if rounded up the result is 4.1, and based on table 5 the results are obtained with the satisfied category.

Control and Security
The test results on performance indicators are poured into Table 12, and then data processing is carried out using Formula (1), so that the following results are obtained: Based on formula (1), the test results in Table 12 obtained the following calculation JSK (total number of overall scores) is 857 and JK (total number of respondents' answers) is 200.
So that the value of RK is 857 divided by 200 is 4.285 if rounded up the result is 4.3, and based on table 5 the results are obtained with the very satisfied category.

Efficiency
The test results on performance indicators are poured into Table 13, and then data processing is carried out using Formula (1), so that the following results are obtained: Based on Formula (1), the test results in table 13 obtained the following calculation JSK (total number of overall scores) is 1282 and JK (total number of respondents' answers) is 300.
So that the value of RK is 1282 divided by 300 is 4.273 if the rounded up the result is 4.3, and based on table 5 the results are obtained with the very satisfied category.

Services
The test results on performance indicators are poured into Table 14, and then data processing is carried out using Formula (1), so that the following results are obtained: Based on Formula (1), the test results in Table 14 obtained the following calculation JSK (total number of overall scores) is 1693 and JK (total number of respondents answers) is 400.
So that the value of RK is 1693 divided by 400 is 4.2345 if the rounded up the result is 4.2, and based on Table 5, the results are obtained in the very satisfied category.

Recapitulation of Test Results
The overall recapitulation based on the above calculations is presented in Table 15. Based on the results above, there is one point generated that has a different category from all the testing indicators, namely the economics indicator. So that it needs to be improved again in terms of the information displayed so that it has more use value to be reused, besides that the use of the system can provide convenience for users so that system users can solve problems without the need to make visits, so as to reduce operational costs in the process of procuring goods and services.
Advice that can be given to website managers in terms of the information displayed is information that can be useful in the short and long term about the upcoming procurement of goods and services, contact persons, announcements about system maintenance, so that users get information to minimize confusion for users. In addition, business processes with a system can minimize interaction between system managers and users, thereby saving user expenses in terms of making visits or coordination.

Conclusion
Evaluation of the goods/services procurement website for the city of Denpasar using the PIECES method is intended to determine website performance. Performance testing is intended to maintain system consistency that is accessed by many users, so that system quality is always well maintained. This study uses 22 questions and 100 respondents consisting of providers and government employees who use the system. The results of the tests that have been carried out in this study, show that the performance indicators get a satisfaction value of 4.2 which is included in the very satisfied category, the information indicator with a value of 4.1 is included in the satisfied category, the economics indicator with a value of 4.2 is in the very satisfied category, the control indicator and security with a score of 4.2 in the very satisfied category, the efficiency indicator got a score of 4.3 and entered into the very satisfied category, and the services indicator with a value of 4.0 in the satisfied category. Overall, the test results state that the website for the procurement of goods and services for the city of Denpasar, which is located at https://eproc.denpasarkota.go.id, has good performance so that users are very satisfied with the performance of the existing system. However, it needs to be improved on information and services indicators so that the system is even better for serving the procurement of goods and services in the City Government of Denpasar. Advice that can be given to website managers in terms of the information displayed is information that can be useful in the short and long term about the upcoming procurement of goods and services, contact persons, announcements about system maintenance, so that users get information to minimize confusion for users. In addition, business processes with a system can minimize interaction between system managers and users, thereby saving user expenses in terms of making visits or coordination.  The winning information from the procurement or tendered work, which is displayed on the website eproc.denpasarkota.go.id has detailed information INF03

REFERENCE
Information on the schedule of stages of the procurement or work being tendered, which is displayed on the website eproc.denpasarkota.go.id has detailed information

INF04
The existence of an electronic-based procurement system reduces your costs in making or participating in a job tender in the Denpasar City government INF05

Economics
The eproc.denpasarkota.go.id website provides information about the amount of the budget in each job ECO01 The eproc.denpasarkota.go.id website provides information about the bid value of the job winners ECO02 The eproc.denpasarkota.go.id website provides information about the costs involved in bidding on a job ECO03

Control and Security
There are access restrictions when you log in to the eproc.denpasarkota.go.id website, different users for menu access for provider and non-provider user types

SEC01
The data in the system can be guaranteed confidentiality SEC02 Efficiency With the eproc.denpasarkota.go.id website, it can provide operational cost savings in carrying out procurement preparations or making bids on one of the jobs

EF01
Through the website eproc.denpasarkota.go.id, you can use this website to make work or make job offers for all types of procurement EF02 With the eproc.denpasarkota.go.id website, it can provide convenience in completing work in the procurement of goods and services EF03

Services
Contacts listed on the Website eproc.denpasarkota.go.id can be contacted SRV01 The data filter and search features found on the eproc.denpasarkota.go.id website display the appropriate data SRV02 If there is a problem, the service provided from the contact displayed on the eproc.denpasarkota.go.id website page provides a quick response SRV03 You, as a company or procurement agent in a Denpasar city government agency, who have attended or are currently participating in a procurement process, are satisfied with the website eproc.denpasarkota.go.id SRV04