Internet Related Technologies in the auditing profession: A WOS bibliometric review of the past three decades and conceptual structure mapping

ABSTRACT

Research on Internet-Related Technologies in the auditing profession has grown substantially over the past three decades; however, it is very fragmented. This study seeks to synthesize and provide a comprehensive overview of the literature. Using bibliometric techniques and content analysis, this study provides an exhaustive overview of the research on Internet-Related Technologies in the auditing profession. The study utilized bibliography from the Web of Science database spanning for three decades from 1990 to 2019. A total of 236 academic documents, written by 478 authors from 102 sources was retrieved and used for the analysis. HistCite and Biblioshiny in R were used to run the citation and network analysis. Influential journals, institutions, trending articles and important network collaborations were identified. Bibliographical coupling in a data visualization software (VOSviewer) and content analysis in Excel were used to identify the following six major research streams: (1) the use of big data analytics in the auditing profession, (2) impact of Internet Related Technologies on continuous auditing, (3) impacts of Internet Related Technologies on audit quality and efficiency (4) impact of Internet Related Technologies on fraud detection and risk assessment (5) blockchain and the auditing profession (6) cloud auditing and audit support systems. Potential important research avenues and practical implications of the research were also provided.

Tecnologías relacionadas con Internet en la profesión de auditor: Una revisión bibliométrica de la WOS de las últimas tres décadas y un mapa de la estructura conceptual

RESUMEN

La investigación sobre las tecnologías relacionadas con Internet en la profesión de auditor ha crecido sustancialmente en las últimas tres décadas; sin embargo, está muy fragmentada. Este estudio pretende sintetizar y ofrecer una visión global de esa literatura. Utilizando técnicas bibliométricas y de análisis de contenido, este estudio proporciona una visión exhaustiva de la investigación sobre las tecnologías relacionadas con Internet en la profesión de auditor. El estudio utilizó bibliografía de la base de datos Web of Science que abarca tres décadas, desde 1990 hasta 2019. Se recuperaron y utilizaron para el análisis un total de 236 documentos académicos, escritos por 478 autores de 102 fuentes. Se utilizaron HistCite y Biblioshiny en R para ejecutar el análisis de citas y redes. Se identificaron las revistas influyentes, las instituciones, los artículos de tendencia y las colaboraciones importantes en red. Se utilizó el acoplamiento bibliográfico en un software de visualización de datos (VOSviewer) y el análisis de contenido en Excel para identificar las siguientes seis grandes corrientes de investigación: (1) el uso de big data en la profesión de auditoría, (2) el impacto de las tecnologías relacionadas con Internet en la auditoría continua, (3) los impactos de las tecnologías relacionadas con Internet en la calidad y la eficiencia de la auditoria, (4) el impacto de las tecnologías relacionadas con Internet en la detección del fraude y la evaluación del riesgo, (5) blockchain y la profesión de auditoria y (6) la auditoría en la nube y los sistemas de apoyo a la auditoría. También se proporcionaron posibles vías de investigación importantes e implicaciones prácticas de la investigación.

1. Introduction

The internet revolution and new advances in technology over the past three decades have enabled the adoption of several novel technologies within the auditing profession. Technologies such as cloud auditing have enabled auditors to process information online (DaSilva et al., 2013; Hu et al., 2018), facilitating the provision of auditing and assurance services on a real-time and continuous basis (Eulerich & Kalinichenko, 2018; Vasarhelyi & Halper, 1991). Additionally, blockchain technology and their smart contracts are increasing auditability of transactions (Dai & Vasarhelyi, 2017; O'Leary, 2017; Yermack, 2017) whilst artificial intelligence is still useful in audit decision making situations (Sutton et al., 2016). Furthermore, unstructured internet related big data sources such as social media are complimenting traditional sources of audit evidence (Appelbaum, 2016; Huerta & Jensen, 2017; Yoon et al., 2015). All these technologies are becoming more relevant to the audit process (Cong et al., 2018; Dai, 2017). Therefore, the definition of Internet Related Technologies “IRT” adopted in this study is similar to that of (Moll & Yigitbasioglu, 2019) which includes technologies which might or might not necessarily depend on the internet such as big data, artificial intelligence, cloud auditing, blockchain, continuous auditing and audit support systems which might be useful to auditors.

These IRTs are improving audit efficiency and effectiveness (Bierstaker et al., 2014; Bierstaker et al., 2001; Curtis & Payne, 2008). For instance, auditors can now provide audit and assurance services on unstructured data (Richins et al., 2017), test almost all financial and nonfinancial transactions (Cao et al., 2015; Kogan et al., 2014) and navigate faster in large data sets (Gepp et al., 2018). They are changing the way auditors work and has the potential to be disruptive innovations to the profession (Earley, 2015).

Although the auditing profession seems to be lagging behind in the adoption of these technologies (Alles, 2015; Austin et al., 2018; Earley, 2015; Gonzalez et al., 2012; Salijeni et al., 2019), the past few years have witnessed an increasing interest in them. For instance, minor discussions on Big Data Analytics in the auditing profession commenced around 2012 (Salijeni et al., 2019). In 2015, these discussions gathered momentum (Hampton & Stratopoulos, 2016). Since then, Big Data Analytics and other IRTs have been important phrases within the profession (Favaretto et al., 2019; Moll & Yigitbasioglu, 2019). As a result of this, there has been a rapid growth in publications (Figure 1), increase in conferences, presentations and special issues in some journals (Alles & Gray, 2016; Hampton & Stratopoulos, 2016; Janvrin & Watson, 2017; Moll & Yigitbasioglu, 2019).

By using quantitative (bibliometrics) techniques, this study aims to explain how research on IRT in the auditing profession has evolved over the past three decades. Existing literature reviews have tried to synthesize this body of knowledge qualitatively mainly focusing on Big Data Analytics (Appelbaum et al., 2017; Appelbaum et al., 2018; Cockcroft & Russell, 2018; Gepp et al., 2018), Continuous Auditing (Brown et al., 2007; Chiu et al., 2014; Eulerich & Kalinichenko, 2018; Kuhn Jr & Sutton, 2010; Murcia et al., 2008), Artificial Intelligence on a broader perspective of accounting research (Sutton et al., 2016) and Moll and Yigitbasioglu (2019) reviewed a number of these technologies in auditing, management accounting and financial accounting. In contrast, this bibliometric approach is methodologically different to all the above-mentioned studies. It provides a different and unique dimension of reviewing literature objectively using well-grounded analytical methods (Aria & Cuccurullo, 2017). If properly carried out, bibliometric reviews can be excellent complements to the above mentioned studies in synthesizing literature on technology adaption in the auditing profession (Guo et al., 2019).

To the best knowledge of the author, a few bibliometric studies are available on related literature (Ardianto & Anridho, 2018; Chiu et al., 2019; Marques & Santos, 2017; Muehlmann et al., 2015). Chiu et al. (2019) provided a general overview of the methodologies used in accounting information systems journals in general. Ardianto & Anridho (2018) focused on the methodologies used in publications in one specific journal, the International Journal of Digital Accounting Research. Whilst, Muehlmann et al. (2015), provided an overview of the publications in the first ten years for the Journal of Emerging Technologies in Accounting. Marques & Santos (2017) focused on continuous auditing with their analysis providing number of publications per year by document type (articles, conference papers, books etcetera), number of publications and country of affiliation of the authors, number of publications by scientific area (decision science, economics, social science etcetera).

None of these bibliometric studies presented a detailed overview of this literature within the auditing profession, that is, the impactful journals, influential articles, and schools of excellence. This led to the study’s first research question: What are the key journals, influential institutions, authors, impactful and trending articles, and influential collaborations in Internet Related Technologies within the auditing profession? Secondly, a significantly high volume of research in this domain has been published over the past years (Figure 1). This spontaneous growth in publication numbers has made it a strenuous task to keep track of how the research field has evolved over the past three decades. Thus, the study endeavors to synthesize this body of knowledge utilizing a unique quantitative approach and reveal the intellectual structure within it. Therefore, the second research question for the study is: What are the underlying research streams in Internet Related Technologies literature and which of them needs further research attention?

The first question provides the classical articles and impactful perspectives which are worth reading for further advancement of knowledge in IRT. It also reveals possible journals that researchers should consider publishing their research in the future. Further, it provides authors and institutions which are leading in this domain, probably giving some insights into possible future collaborations between emerging institutions/authors and the leading ones and possible employment opportunities in the future for faculty. The second question shows us the key research streams within this area. This will provide an overview of what has been researched over the past three decades, furnishing researchers and audit practitioners with a comprehensive guide where they can quickly find informative references on IRT. It will also avail tensions and research gaps within this body of knowledge which might need further attention for meaningful advancement of knowledge in IRT.

This paper is organized as follows: section 2 presents the research methodology followed with section 3 presenting bibliometric findings. Section 4 then presents an in-depth discussion of the research streams in IRT literature and section 5 concludes the study by providing research implications and future research agenda.

2. Research Methodology

2.1. Literature Search

To answer the research questions stated in Section 1 above, this study utilized a sample of research publications collected by the Web of Science “WOS”, a database which has been utilized by other published bibliometrics studies in highly reputable journals (Apriliyanti & Alon, 2017; Munim et al., 2020; Uyar et al., 2020). The WOS has a database of publications spanning as far back as 1945 and consists of data from over 3000 high quality peer reviewed journals in social sciences (Alon et al., 2018). As of July 2020, the WOS database consisted of over 21 000 peer reviewed journals, over 205 000 conference proceedings and over 104 000 books (from clarivate.libguides.com). Even though its dominance is being challenged by Scopus, WOS is still the widely used database in academic papers (Zhu & Liu, 2020). However, the sample used in this study included publications from 1990 to 2019. The year 1990 is a suitable starting point since internet become popular on a world wide scale through World Wide Web in the early 1990s (Marson, 1997; Segal, 1995). Secondly, discussions about early IRT adoption in the auditing profession commenced around 1990 (see Vasarhelyi & Halper, 1991).

To ensure an exhaustive coverage of the literature, a seven-step data collection approach with twenty-one search terms using a Boolean function was used (Table 1). The first part ("digitization*"OR "digitalization*"OR "artificial intelligence" OR "business intelligence*" OR "automation*" OR "big data*" OR "data analytics*" OR "blockchain*" OR "continuous audit*" OR " continuous monitoring*" OR "continuous assurance*"OR "visualization*" OR "information technology*"OR "machine learning*" OR "cloud¹*"OR "intelligent systems" OR "knowledge-based systems" OR "intelligent decision aid" OR " intelligent decision support system" OR "Internet of things") was capturing the IRT literature, whilst AND (audit*) was capturing the auditing profession. These search terms are similar to those adopted by (Munim et al., 2020). To ensure that the term (audit*) was enough to capture the whole profession, the author used ("audit" OR " auditor" OR "auditing" OR "auditors" OR "auditing profession " OR "audit profession") but couldn’t find any significant difference from just using (audit*). Therefore, a decision to use (audit*) alone was viewed as sufficient to capture the auditing profession. A seven-step procedure as depicted in (Table 1) was carefully carried out.

Table 1. Keyword search

	Search Terms Applied	Number of publications
Step	Words used
1	"audit*" AND ("big data" OR "data analytics")	72
2	"audit*" AND ("artificial intelligence " OR "business intelligence" OR "blockchain")	39
3	"audit*" AND ("automation” OR "machine learning" OR "visualization*")	32
4	"audit*" AND ("continuous monit*" or "continuous audit*")	93
5	"audit*" AND ("digitization*" OR "digitalization*" OR "information technology") After screening through reading the abstracts of the articles found, 147 articles were selected. However, a decision was made to add conference proceedings and more keywords synonyms such as “cloud*, internet of things, expert systems, intelligent systems, knowledge-based systems, intelligent decision aid, intelligent decision support system” on the Boolean search function.	131
6	"audit*" AND("digitization*"OR "digitalization*"OR "artificial intelligence" OR "business intelligence*" OR "automation*" OR "big data*" OR "data analytics*" OR "blockchain*" OR "continuous audit*" OR " continuous monitoring*" OR "continuous assurance*"OR "visualization*" OR "information technology*"OR "machine learning*" OR "cloud*"OR "intelligent systems" OR "knowledge-based systems" OR "intelligent decision aid" OR " intelligent decision support system" OR "Internet of things")	393
7	After reading titles, abstracts and quick scan of the 393 documents, a final sample of 236 documents was selected	236

After thoroughly reading through the titles, abstracts and quick scan of the 393 publications, a final sample of 236 publications was deemed suitable for use in the study. This sample was from 102 sources, written by 478 authors and connected to 275 institutions. It comprised of 161 journal articles (68.2%), 55 proceedings papers (23.3%) and 20 editorial material (8.5%). Only 47 publications were single authored. The sample has an average of 2.03 authors per document. Conference proceedings were included because the topic is still trending and by including the proceedings, the study will capture all the relevant studies on IRT in the auditing domain (Munim et al., 2020). To control for adequacy of the dataset retrieved from WOS, the author scanned through a number of published systematic literature reviews in IRT e.g. (Appelbaum et al., 2017; Cockcroft & Russell, 2018; Eulerich & Kalinichenko, 2018; Gepp et al., 2018; Moll & Yigitbasioglu, 2019) to compare if there are any significant articles and insights used in those studies which might not have been retrieved from WOS for this current study. The author was satisfied that the sample of 236 was an adequate representation of the articles in IRT literature in the auditing profession.

2.2. Bibliometric Analysis

The 236 publications were analyzed using bibliometric techniques and content analyses. Bibliometric techniques are quantitative in nature and are used to synthesize literature in a replicable manner (Maditati et al., 2018; Zupic & Čater, 2015). It uses the article as the basic unit of analyses. There are various bibliometric methods, but the current study primarily used citation analysis, network analyses, bibliographical coupling (citation mapping technique) and content analysis. Citation analysis was used to show the performance of journals, articles, institutions and authors (Zupic & Čater, 2015). The study used Total Global Citations “TGC”, Total Local Citations “TLC” and Total Local Citations per year “TLC/t”. TGC represents the total number of citations a study receives in the whole WOS database while TLC represents the number of times a publication has been cited by other publications within the current sample of 236 publications (Alon et al., 2018). If the TLC is controlled for time, then it becomes TLC/t. Network analysis was used to show the collaboration among institutions authors and how key words have evolved over the past years. Bibliographical coupling was used to map the intellectual structure of IRT literature in auditing. It was chosen over other methods because bibliographical coupling is most applicable to mapping current research fronts (Aria & Cuccurullo, 2017). The citation analysis, network analysis, bibliographical coupling (citation mapping technique) and content analysis were carried out using Histcite, Biblioshiny in R, VOSviewer and Excel respectively (Table 2).

For question one, the study relied on citation analysis and network analysis in HistCite and Biblioshiny in R. For research question two, bibliographical coupling was used coupled with content analysis. Out of the 236 publications, 89 publications which were highly bibliographically coupled to each other were used to identify underlying research streams in VOSviewer. VOSviewer was chosen for the intellectual mapping because one can elucidate its visuals effortlessly (Aria & Cuccurullo, 2017). Excel was used for content analyses. As much as there are trending data mining tools which could have been used for this, Excel was chosen because it enables the author to have more control of the article and pluck more sense out of it through a detailed concept matrix (Maditati et al., 2018). A concept matrix is typically comprised of important information from the article such as title, year of publication, its authors, research questions, findings and the research stream it falls under (Maditati et al., 2018). A quick scan of the whole sample was also conducted to make sure that some articles which were not highly bibliographically coupled but have significant insights were included in the content analysis. This enables the author to conduct a thorough literature synthesis and document all the relevant insights from documents in the sample.

Table 2. Software, use and outcome

Software	Use	Outcome
HistCite	Citation analysis	Trending articles, Influential sources Influential authors Schools of Excellence
Biblioshiny in R	Sankey diagrams and network analysis	Annual scientific production University collaborations Authors collaborations Three field plots Word Growth
VOSviewer	Bibliographical Coupling	Mapping of the intellectual structure in IRT literature
Excel	Content analysis	Content analysis on intellectual structure

3. Bibliometric findings

3.1. Annual Production

There has been an increasing interest in IRT within the auditing profession over the past three decades with an annual average growth rate of 12.1% (Figure 1). Notably, the past 5 years 2015 to 2019 witnessed a sharp increase in annual production of IRT literature. As a result of this, almost 30.5% of the documents in the sample were produced during the first 25 years of the 30 under review, with almost 69.5% published during the last 5 years. This might be because the auditing profession opened more to these technologies around that time, with some journals issuing out special issues and an increase in conference proceedings on these technologies (Alles & Gray, 2016; Hampton & Stratopoulos, 2016; Janvrin & Watson, 2017; Moll & Yigitbasioglu, 2019).

Figure 1. Annual production

3.2. Most Influential Publications

Gaining an overview of the most influential articles in a research domain helps to identify classical articles with most impactful perspectives and probably predict potential areas of future research (Alon et al., 2018). The study identifies most influential articles by using the Total Local Citations adjusted for time (TLC/t) as shown in (Table 3). Articles related to big data and the auditing profession were identified as the most influential articles in IRT literature. Four of the top five trending articles are in that research domain. Interestingly, most of these articles were released in the same special issue in Accounting Horizons in 2015. Of the 10 most influential articles in IRT, 6 are from that same special issue.

Table 3. Trending articles in IRT literature in auditing profession

Rank	Article	Journal*	TLC/t	TLC	TGC
1	(Appelbaum et al., 2017)	AAJPT(3)	4.67	14	31
2	(Dai & Vasarhelyi, 2017)	JIS(1)	4.00	12	64
3	( Vasarhelyi et al., 2015)	AH(3)	3.80	19	86
4	(Cao et al., 2015)	AH (3)	3.80	19	51
5	(Yoon et al., 2015)	AH (3)	3.80	19	47
6	(Alles, 2015)	AH(3)	3.60	18	41
7	(Issa et al., 2016)	JETA(1)	3.25	13	24
8	(Moffitt et al., 2018)	JETA(1)	3.00	6	11
9	(Krahel & Titera, 2015)	AH(3)	2.80	14	40
10	(Zhang et al., 2015)	AH(3)	2.6	13	38

TLC/t Average local citations received per year; TGC global citations received

* For abbreviations of journal names see Table 4 below. Journal ranking in parenthesis is based on ABS 2018 journal guide.

3.3. Most Influential Journals/ Academic sources

The 236 publications used in this study were from 102 academic sources. These academic sources were ranked using the number of IRT publications “P_IRT” and the Total Local Citations received in the sample (TLC). The top three influential journals ranked by number of IRT publications “P_IRT” (Table 4) are Journal of Emerging Technologies in Accounting, International Journal of Accounting Information Systems and Journal of Information Systems. Interestingly, all have an IRT and or information system focus. The three represent almost 30% of the total publications in the sample. Accounting Horizons and Auditing- A-Journal of Practice &Theory completed the top 5 respectively. When measuring influence by the number of citations received by the journals, Accounting Horizons, Auditing- A-Journal of Practice &Theory, International Journal of Accounting Information Systems, Journal of Information Systems and Journal of Emerging Technologies in Accounting complete the top 5 journals in that order (Table 4).

Using the Association of Business School (ABS) journal ranking, the top ranking journals (ranked by P_IRT) is dominated by journals from ABS level 1 and 2. However, when we rank the journals using citations received (TLC), the influence of level 3 journals going upwards increases on the list (Table 4). This might suggest that IRT literature is mainly concentrated in lower ranked journals, but highly ranked journals have more influence on citations.

Table 4. Most Influential Sources

Rank	Source	ABS level	TLC	TLC/t	TGC	TGC/t	% of PIRT	Source	ABS level	PIRT
1	AH	3	137	29.00	485	92.87	6.4	JETA	1	29
2	AAJPT	3	82	9.96	386	34.73	5.9	IJAIS	2	22
3	IJAIS	2	53	9.53	209	42.29	9.3	JIS	1	19
4	JIS	1	48	12.60	165	51.48	8.1	AH	3	15
5	JETA	1	44	15.08	103	32.97	12.3	AAJPT	3	14
6	AR	4*	30	3.12	147	17.43	1.3	MAJ	2	12
7	CAR	4	11	2.67	32	10.33	1.3	CIA	2	5
8	MAJ	2	10	3.05	36	10.90	5.1	AJBM	-	4
9	AOS	4*	6	0.25	44	1.83	0.4	AR	4*	3
10	BH	2	6	1.20	14	2.80	0.4	CAR	4	3
11	JAL	3	6	3.00	36	18.00	0.8
12	RF	4	6	2.0	96	32.00	0.4

Key: Journal of Emerging Technologies in Accounting- JETA, International Journal of Accounting Information System- IJAIS, Journal of Information System- JIS, Accounting Horizons- AH, Auditing- A-Journal of Practice &Theory- AAJPT, Managerial Auditing Journal- MAJ, African Journal of Business Management- AJBM, Current Issues in Auditing- CIA, Accounting Review- AR, Contemporary Accounting Research- CAR, Journal of Accounting Literature- JAL, Accounting Organization & Society- AOS, Business Horizons- BH, Review of Finance- RF.

3.4. Schools of Excellence

As shown in (Table 5), Rutgers State University (USA) is the leading institution in terms of quantity of publications contributing almost 15% of the total publications in the sample. Rutgers State University has a Continuous Audit and Reporting Laboratory (CarLab) which is dedicated to research on IRT in the auditing profession. Bucharest University of Economic Studies (Romania), Southwestern University of Finance & Economics (China), University of Central Florida (USA), University of Arkansas (USA) and Villanova University (USA) completed the top six institutions. When measuring the influence of institutions using TGC, Rutgers State University still leads the pack. However, most of the contributions from Bucharest University of Economic Studies were conference proceedings which has greatly affected its TLC and TGC.

Table 5. Schools of Excellence

Rank	Institution	Country	PIRT	Percentage	TLC	TGC
1	Rutgers State University	USA	35	14.80	226	681
2	Bucharest University of Economics Studies	Romania	7	3.00	1	2
3	Southwestern University of Finance & Economics	China	6	2.50	19	87
4	University of Central Florida	USA	6	2.50	16	40
5	University of Arkansas	USA	5	2.10	22	103
6	Villanova University	USA	5	2.10	19	126
7	Eastern Illinois University	USA	4	1.70	11	15
8	Iowa State University	USA	4	1.70	18	69
9	Montclair State University	USA	4	1.70	17	40
10	University of Memphis	USA	4	1.70	22	87
11	University of Texas	USA	4	1.70	2	24

Additionally, Figure 2 shows the collaborations among universities. Collaborations of universities in the top left corner is dominated by Rutgers State University working in partnership with mostly other American universities. However, there are some interesting collaborations between American universities and universities from other countries such as China (Xian Jiaotong- Liverpool University, Nanjing University, Southwestern University of Finance and Economics,), United Kingdom (University of Manchester) and Italy (University of Parma and Free University of Bozen-Bolzano). The overall results on Schools of Excellence showed the dominance of Rutgers State University and USA as a country in this domain

Figure 2. University collaborations

Figure generated using normalization: none, network layout: automatic, clustering algorithm: walk trap and number of nodes: 50

3.5. Influential Authors

Table 6 shows the most influential authors in IRT in the auditing profession over the past three decades ranked by Total Local Citations adjusted for time (TLC/t). The table suggests that Vaserhelyi, Kogan and Alles are the three most influential researchers in IRT in the auditing profession in that order. Interestingly, Vaserhelyi is a Distinguished Professor of Accounting at Rutgers State University and Director of the Continuous Audit and Reporting Laboratory (CarLab). The list for most influential authors is mainly dominated by researchers who are or were once affiliated to Rutgers State University, some of whom have completed their PhD studies under the Directorship of Vaserhelyi. Additionally, Figure 3 shows the authors collaborations. The figure further shows the influence Vaserhelyi have in this domain as depicted by the large red box he occupies. It shows that he mainly collaborates with other faculty members from the same research institute and the US at large.

Table 6. Influential Authors

Rank	Author	TLC/t	TLC	TGC/t	TGC
1	Vasarhelyi MA	27.42	122	81.82	361
2	Kogan A	15.75	73	48.75	222
3	Alles MG	11.79	80	27.01	195
4	Appelbaum D	10.02	37	22.52	85
5	Sun T	6.05	19	9.80	31
6	Dai J	4.75	15	22.83	70
7	Zhang L	4.47	21	10.07	49
8	Rozario AM	4.00	7	7.50	13
9	Cao M	3.80	19	10.20	51
10	Chychyla R	3.80	19	10.20	51
11	Hoogduin L	3.80	19	9.40	47
12	Stewart T	3.80	19	10.20	51
13	Tuttle BM	3.80	19	17.20	86
14	Yoon K	3.80	19	9.40	47

TLC/t Average local citations received per year; TLC Total local citations received; TGC/t Average global citations received per year; TGC Total global citations received

Figure 3. Authors collaboration

Figure generated using normalization- none, network layout- automatic, clustering algorithm - walk trap, number of nodes – 50, minimum edge- 0.

3.6. Three- Field Plot, Word Growth, and key word evolution over the three decades

Based on the Sankey plot, the study presents the three-field plot (Figure 4). A three-field plot is a Sankey diagram which shows the interactions amongst different fields (Riehmann et al., 2005). To obtain a clearer diagram, the Sankey plot was plotted using five countries (right), five most influential journals (left) and five most recurring author’s keywords (middle) within the IRT literature. Author’s key words in the middle represents the topic of the study whilst the most influential journals on the left shows the source of the IRT literature interested in the keywords. The country on the right shows the country of affiliation of the institutions and authors who have been working on the keywords in the middle. On an aggregate, all countries and journals used in the plot are generally interested in the topics involving the five key words used for the analysis. The figure further shows that since its inception in the early 1990s, continuous auditing is still a popular term in IRT literature. Big data and data analytics are other keywords which authors are more interested in. This can also be shown in the word growth plot for the past three decades (Figure 5).

Figure 4. Three Field Plot

Plotted using source documents, author’s keywords, and country.

Figure 5. Word growth using author's keywords

To show how the key words in IRT literature have evolved over the past thirty years, the study used the dynamic co-occurrence network in Biblioshiny in R. The three decades were subdivided into 10-year periods (Figure 6.1, Figure 6.2, and Figure 6.3). As shown in (Figure 6.1), the period 1990 to 1999 was mainly dominated by how IRT might support auditor’s judgment decision making. This can be witnessed by the popularity of terms like expert systems, decision support system and decision aid reliance. The next decade 2000 to 2009 was mainly dominated by continuous auditing and monitoring (Figure 6.2). As will be discussed in section 4.5.1 below, the adoption of continuous auditing was mainly driven by the accounting scandals which took place during the turn of the millennium and subsequent introduction of Sarbanes-Oxley (SOX) Act (Alles et al., 2002; El‐Masry & Reck, 2008). Terms related to supporting auditor’s decision-making process such as expert systems were still around but not as dominant as they were in the previous decade. The following decade 2010 to 2019 saw a plethora of key IRT words emerging in research such as blockchain, big data, cloud computing, machine learning and data analytics (Figure 6.3). As previously noted in the word growth analyses presented in (Figure 5), continuous auditing continued to be one of the key terms from the previous decade. Artificial intelligence related terms such as expert systems were no longer as dominant as they were during the previous decades.

Figure 6.1. The co-occurrence network of author's key words for period 1990- 1999

Figure generated using normalization- none, network layout- automatic, clustering algorithm - walk trap, number of nodes – 50, minimum edge- 1.

Figure 6.2. The co-occurrence network of author's key words for period 2000- 2009

Figure generated using normalization- none, network layout- automatic, clustering algorithm - walk trap, number of nodes – 50, minimum edge- 1.

Figure 6.3. The co-occurrence network of author's key words for period 2010- 2019

Figure generated using normalization- none, network layout- automatic, clustering algorithm - walk trap, number of nodes – 50, minimum edge- 1.

4. Research streams

To identify the underlying research streams, the study used bibliographic coupling technique in VOSviewer. To improve the graphical presentation of the conceptual map, a minimum of 5 citations of a document was selected as the threshold for documents to be considered for bibliographic coupling. Out of the sample of 236 documents, 89 met this criterion and were used to identify underlying research streams. Six research streams were found in IRT literature (Figure 7) namely: (1) the use of big data analytics in the auditing profession, (2) impact of Internet Related Technologies on continuous auditing, (3) impacts of Internet Related Technologies on audit quality and efficiency, (4) impact of Internet Related Technologies in fraud detection and risk assessment, (5) blockchain and the auditing profession, (6) cloud auditing and audit support systems.

Figure 7. Research Clusters

Plotted using: Type of analyses - Bibliographic coupling, Unit of analyses - Documents, Counting Method- Full counting, Minimum number of citations of a document -5, Minimum number of articles per cluster -5.

4.1. Big Data Analytics “BDA” in the audit profession

Big data can be defined in various ways (Yoon et al., 2015). Generally, it refers to huge sets of data with large, diversified, and complex structure, which is difficult to store, analyze and conceptualize for further processing. Its sources are both internal and external to the organization. They are generated from diversified sources such as social media (Appelbaum et al., 2017). Big data is characterized by three “Vs” namely volume, velocity and variety (Cao et al., 2015) and some scholars often suggest two more “Vs” veracity and value (Janvrin et al., 2017). For big data to be useful in the audit process, there is need to explore it further and pluck out audit relevant information. Analysis of big data requires variegated complex technological tools from different disciplines- data analytics (Cao et al., 2015). Therefore, big data and data analytics are two highly related terms which are independent from each other (Alles & Gray, 2016). Big Data Analytics “BDA” tries to explain the relationship between these two variables (Cao et al., 2015; Richins et al., 2017; Salijeni et al., 2019).

4.1.1. Drivers for the use of BDA in the auditing profession

Drivers for the adoption of BDA in the auditing profession can be categorized into internal and external drivers(Michael G. Alles, 2015). Generally, audit client’s level of BDA technology adoption determines the level of BDA tools to use during audits of financial statements (Dagilienė & Klovienė, 2019; Lombardi et al., 2014). If there is a discrepancy in the level of technology between auditors and their clients, it might have an effect on the audit opinion issued (Salijeni et al., 2019). Therefore, the auditors’ need to strike a balance between the technology they have and that of their clients is one of the main external drivers for the adoption of BDA by auditors (Alles, 2015).

One internal driver which is related to the previously mentioned external driver is the need for auditors to embrace technological tools which will provide better audit insights and ultimately improve audit quality (Alles & Gray, 2016; Earley, 2015; Salijeni et al., 2019). A number of literature have documented that BDA improves audit quality (Brown-Liburd et al., 2015; Cao et al., 2015; Vasarhelyi et al., 2015; Yoon et al., 2015). It enhances auditors’ ability to maneuver around massive data, unearth concealed patterns and be able to gather sufficient audit evidence to base their audit opinion on (Yoon et al., 2015). Additionally, BDA improves insights from auditors leading to a better client-audit relationship (Earley, 2015). The need for auditors to embrace all these benefits is one of the main internal drivers for auditors to adopt BDA in audits of financial statements.

Despite these important drivers, the rate of adoption for BDA in the auditing profession has not been as wide spread as in other professions (Alles, 2015; Earley, 2015; Gepp et al., 2018; Griffin & Wright, 2015). There are various reasons, challenges, inhibitors and or barriers which have been suggested for that.

4.1.2. Reasons for slow adoption of BDA in the auditing profession

One common reason for the slow adoption of BDA into the auditing profession is the lack of skills required to work with complicated BDA related tools effectively (Alles, 2015; Austin et al., 2018; Earley, 2015; Salijeni et al., 2019). BDA is associated with large volumes of data which are complex in nature. Maneuvering in these data types using sophisticated BDA tools requires mastery of an idiosyncratic set of data science knowhow (Huerta & Jensen, 2017). Unfortunately, the majority of the auditors do not possess these unique skills (Salijeni et al., 2019). They are not normally taught in accounting and auditing courses in business schools, but auditors usually gain them through rigorous on the job trainings, many years of experience and intensive data science related courses.

One way to deal with data science skills deficiency amongst auditors would be to invest in training sessions. Regrettably, these training costs are expensive and out of the reach for many, especially small audit firms (Dagilienė & Klovienė, 2019). Alternatively, auditing firms can hire data scientists who already possess the necessary skills to deal with the intricate BDA tools. One main drawback of this option is that these experts might not have similar audit training as the traditional auditors which might consequently negatively affect audit quality(Alles & Gray, 2016). Additionally, Salijeni et al. (2019) empirically found that in some audit teams with both data science experts and auditors there were squabbles between auditors and these experts on who should choose the tools to use in an engagement. Thus, these tensions between auditors and data scientists might explain the reasons why there is a slow uptake of BDA in auditing.

An audit firm’s technological proficiency is a function of the technological assets/tools possessed by the audit firm and auditor’s skills (Li et al., 2018). In addition to scarcity of BDA trained auditors, the technological assets are highly priced (Alles & Gray, 2016; Salijeni et al., 2019). As a result, there is a big difference in BDA implementation between the big 4 audit firms- who typically have the financial muscle to commit to costly investments and smaller audit firms- who are typically cash trapped (Dagilienė & Klovienė, 2019). Thus, financial constrains to acquire the tools is one of the reasons why there is a slow adoption of BDA in the audit profession

Another challenge which the auditing profession is currently facing is the incorporation of evidence from BDA tools into the traditional audit evidence (Brown-Liburd & Vasarhelyi, 2015; Yoon et al., 2015). Although the use of large non-financial information is not an entirely new phenomenon in auditing, big data sources are unstructured and highly intricate in nature (Brown-Liburd & Vasarhelyi, 2015). Auditors are not very accustomed to these sources of information (Brown-Liburd et al., 2015; Yoon et al., 2015). Furthermore, unstructured data is inconclusive and susceptible to many interpretations which might have a negative impact on auditor judgement. Zhang et al. (2015) provides a detailed account of these big data shortcomings and their ultimate challenges to the audit process.

Incorporating BDA in auditing financial statements requires auditors to undergo radical changes concerning the unlimited access of client data (Alles & Gray, 2016). In the BDA era, auditors will have access not just to the financial transactions of their clients, but to the whole company data base. The challenges with this scenario is the auditors will have to put in place complex safety control measures for safe storage of the whole client data base (Huerta & Jensen, 2017). On the other hand, the clients themselves might feel unsafe to share the complete company data base with the auditors (Yoon et al., 2015). This might be due to a number of concerns including cyber security (Yoon et al., 2015). Therefore, unusual access to client data might not be a comfortable scenario for both parties leading to the slow adoption of BDA.

4.1.3. Big Data Analytics and audit evidence

Information from BDA can complement traditional financial information from the audit client (Warren et al., 2015; Yoon et al., 2015). BDA applied to reliable big data information, might provide valuable audit evidence since it can be derived from an assortment of sources which are independent from the audit client (Appelbaum, 2016; Huerta & Jensen, 2017; Moffitt & Vasarhelyi, 2013; Yoon et al., 2015). For instance, discrepancies between sales opinions aired on social platforms and the transactions reflected in the sales ledger can be potential areas of high risk in the revenue business process (Murthy & Geerts, 2017; Yoon et al., 2015). Additionally, e-tracking technologies can be used to confirm the delivery statues of products (Moffitt & Vasarhelyi, 2013; Murthy & Geerts, 2017). Thus, BDA information can supplement traditional sources of audit evidence.

Furthermore, some BDA tools such as deep machine learning can unearth hidden insights through text analysis which supports auditor judgment throughout the audit process. They also assist in risk assessment and fraud detection (Earley, 2015; Tang & Karim, 2019). Therefore, by using BDA tools auditors will have more confidence with their opinions even on complicated decisions taken under conditions of uncertainty.

4.2. Impacts of IRT on fraud detection and risk assessments

The turn of the new millennium saw a number of high-profile cases of companies folding due to fraud related accounting scandals (Abbasi et al., 2012; Dong et al., 2018). Since then, the audit profession has been placed under a microscope as auditors were partly held accountable for these scandals. However, the International Standards on Auditing (ISA 240) put the responsibility to prevent financial statements fraud on company’s management and those charged with governance. The auditor’s responsibility is to gather evidence and obtain an assurance that the financial statements do not contain fraud. For a long period of time, auditors have been primarily relying on financial statements information for risk assessment related to fraud (Dong et al., 2018). Although auditors are familiar with financial statements information , fraud detection within those same financial statements has been a time consuming and complex decision making process (Song et al., 2014). The emergence of IRT in auditing has led to the development of decision tools which facilitates financial fraud detection by auditors (Abbasi et al., 2012; Dong et al., 2018; Perols et al., 2017; Song et al., 2014). Furthermore, IRT has also enabled auditors to make informed fraud assessment judgments on complex unstructured data (Dong et al., 2018) comprised of both financial and non-financial risk factors (Song et al., 2014). Thus, IRT has directly addressed previous challenges related to fraud detection in financial statements (Perols et al., 2017).

In addition to fraud detection, auditors are expected to perform risk assessment procedures to assess and identify risk of material misstatement in financial statements in accordance with International Standards on Auditing (ISA 315). Besides it being a tremendous task to carry out, large volumes of data now available to auditors have further complicated this process. Thus the availability of IRT in auditing can assist auditors to address these challenges inherent to complex judgment decision making process in risk assessments (Davis et al., 1997).

4.3. Effects of IRT on audit quality and efficiency

Generally, IRTs are perceived to improve audit quality. They assist auditors in complex judgement decision making situations (Davis et al., 1997) by enabling them to come up with novel creative solutions to problems faced in those situations (Elam & Mead, 1990). Even information technology adopted by the audit client improves both current and future audit quality and efficiency (Pincus et al., 2017). Additionally, Masli et al., (2010) documented evidence which support a positive relationship between internal control monitoring technology and audit efficiency in both internal and external auditing contexts. Actually, external auditors tend to rely more on internal auditor’s work when the internal auditors use IRT technology rather than traditional methods (Malaescu & Sutton, 2015). Thus, the evidence above clearly shows that IRTs can improve audit quality and efficiency.

4.4. Blockchain technology and the auditing profession

Block chain technology is a distributed ledger where transactions are maintained in networked computers (nodes). Blockchain technology is a disruptive innovation which has been predicted to bring radical changes to the auditing and assurance functions (Dai & Vasarhelyi, 2017; Wang & Kogan, 2018). There are two types of blockchains private – which requires permission to join and public-which doesn’t require permission to join (Coyne & McMickle, 2017; Yu et al., 2018). Private blockchains are more secure and most likely to be adopted in various business processes (Coyne & McMickle, 2017; O'Leary, 2017). However, this study is going to generalize both private and public blockchains.

Blockchain technology is built on trust and transparency among the nodes in the chain (Yermack, 2017). Unilateral alteration of information once it has been added to the chain is difficult since multiple similar copies of information are stored by other independent connected nodes (Rozario & Thomas, 2019; Wang & Kogan, 2018). Alteration of information requires validation from other independent nodes. In addition, verification of transactions recorded in the chain is easier since information added to the chain is signed digitally and timestamped (Dai & Vasarhelyi, 2017; Rozario & Thomas, 2019). All these unique features improve auditability and transparency of transactions in the block (Dai & Vasarhelyi, 2017; O'Leary, 2017; Yermack, 2017).

More so, the blockchain network is highly automated and is associated with smart contracts. Smart contracts are artificial intelligent systems which can automatically execute programmed actions traditionally performed through human intervention when certain predefined conditions are triggered (Coyne & McMickle, 2017; Dai & Vasarhelyi, 2017; Yu et al., 2018). As much as they contribute to better information auditability, some fear this high automation might render some audit functions obsolete. Therefore, these idiosyncratic features have made the blockchain innovation to be controversially considered a double-edged sword.

As much as it has been considered a marvelous innovation, there are some few audit concerns about the blockchain technology. Firstly, the adoption of blockchain technology within the audit profession requires highly developed technological resources which some audit firms might not currently possess (Dai & Vasarhelyi, 2017; Yu et al., 2018). These resources include the technological infrastructure and skilled manpower (data science auditors). There are also concerns about striking a balance between information confidentiality and transparency since the more transparency the chain will be, the less confidential the information in it (Coyne & McMickle, 2017; Wang & Kogan, 2018). Moreover, the regulation of information on blockchain is vague, further complicating auditability of financial transactions in the chain (Yu et al., 2018).

Furthermore, there are consistency issues for information recorded in the chain where it might not correspond to the physical world (Liu et al., 2019). In addition to this, blockchain can secure fraudulent information uploaded to the chain. That is, if a deliberate misstatement was uploaded, validated and time stamped, unless discovered that information will be securely stored in the chain. More so, as much as smart contracts can automatically perform certain functions with minimum human interaction, they still lack the judgement decision making element in conditions of uncertainty which is key to the audit process. Therefore, there is still need of a physical human check for transitions taking place on the blockchain technology. These concerned further reiterates the importance of a human auditor in the chain (Liu et al., 2019).

4.5. Impact of IRT on Continuous Auditing

The concept of Continuous Auditing “CA” has existed for almost three decades now , (see Vasarhelyi & Halper, 1991) but still lacks a unanimous definition (Eulerich & Kalinichenko, 2018). Although somehow different, the term continuous auditing can be used synonymously with terms such as continuous monitoring, continuous assurance and real time auditing (Alles et al., 2002). It basically involves the capturing of data, analysis and communicating it to users of financial information on a continuous real time basis (Alles et al., 2002).

4.5.1. Adoption and benefits of continuous auditing

The last three decades have witnessed a widespread use of technology in financial reporting by various audit clients which has resulted in almost real time financial reporting and disclosures. As a result, information is now being availed to users faster than the traditional financial reporting ways (Amin et al., 2016). This has improved information asymmetry. With this type of financial reporting, it has become increasingly difficult to continue applying traditional assurance methods, necessitating the adoption of CA (Elliott, 2002). Additionally, numerous accounting scandals around the turn of the new millennium and their resulting regulations aided to the adoption of CA (Alles et al., 2002). Specifically in the US, CA adoption was mainly driven by the introduction of the Sarbanes-Oxley (SOX) Act (El‐Masry & Reck, 2008), introduced after the previously mentioned accounting scandals.

Once implemented, CA systems can provide reliable information to the auditors utilizing minimum resources (Chen et al., 2007; Malaescu & Sutton, 2015; Rikhardsson & Dull, 2016). Theoretically, CA systems may lead to efficient audits by providing relevant and reliable financial information timeously (Amin et al., 2016; Sun et al., 2015). External auditors seem to rely more on internal audit’s work when they use continuous auditing systems rather than those using traditional systems, further contributing towards efficient audits (Malaescu & Sutton, 2015). Even in Not For Profit making organizations, CA systems increases internal control reliability ultimately improving the image of the organization to its stakeholders (Appelbaum et al., 2016). However, the gains on efficiency can be offset by availability of false positives which require extra manual attention of auditors (Li et al., 2016). More so, the costs of setting up the CA systems is quite high since it is normally associated with high consulting fees (Alles et al., 2002). The cost of infrastructure and the software are quite expensive too (Singleton & Singleton, 2005). In addition to this, the systems are expensive to maintain (Li et al., 2016).

Despite CA being an important component of modern-day auditing, its level of adoption in practice is relatively low and varies significantly across various countries (Gonzalez et al., 2012). For instance, China seems to be lagging the US in CA adoption. Technological gap between these two countries is one of the reasons for the observed discrepancy (Sun et al., 2015).

4.5.2. Impact of big data and blockchain technology on continuous auditing

Big data is associated with four basic Vs, namely volume, veracity, variety, and velocity. This means data now available to CA systems is of high volume, added at a high velocity continuously, with uncertain veracity and of a variety assortment. These four complicated characteristics of big data have brought some data challenges to the current CA systems (Zhang et al., 2015). As a result, the CA systems run a risk of auditing conflicting and incomplete data (Zhang et al., 2015). The current CA systems must manage these challenges for a successful adoption of big data into the current CA systems.

Transactions on a blockchain technology are added on real time basis necessitating the provision of auditing and assurance services on blockchain technology on a continuous manner (Rozario & Thomas, 2019; Wang & Kogan, 2018). Additionally, blockchain technology is normally associated with smart contracts. If properly implemented, smart contacts can create a robust real time auditing system (O'Leary, 2017). They bring with them a number of benefits to CA such as increase in auditability of transactions since it is difficult to alter them once they are real time stamped into the chain (Yermack, 2017). If combined with effective internal controls, they can perform complicated controls, positively contributing towards information auditability in CA systems (Dai & Vasarhelyi, 2017).

4.6. Cloud auditing and audit support systems

4.6.1. Cloud Auditing

Cloud computing is a technology that offers a virtual on demand access to a number of computing resources such as storage, applications and other services (Dillon et al., 2010). In cloud computing, there are user organizations who are the consumers of the cloud services availed by the cloud service providers (Schmidt et al., 2016). Therefore, auditors can provide auditing services to both the users of cloud services and the cloud service providers (Schmidt et al., 2016). Alternatively, they can be cloud service users themselves (Hu et al., 2018). Auditors tend to use cloud auditing services since they reduce costs associated with technological infrastructure acquisitions (Chou, 2015). Additionally, cloud based technologies are an important element in providing auditing services on real time continuous basis (Cong et al., 2018).

Despite these important factors for the adoption of cloud auditing, there are some concerns among auditors regarding the use of cloud technology. Information confidentiality and data privacy has been major concerns among auditors especially towards public cloud service platforms (Yigitbasioglu, 2015). Auditing cloud data can be difficult for auditors since the cloud providers can restrict the amount of information auditors can access due to data privacy issues of other parties using the same cloud platform (Chou, 2015). In addition, in most cases, the physical location of the data is unknown and or is in another legal jurisdiction which might complicate forensic investigations (Yigitbasioglu, 2015). Despite these concerns, many auditors are adopting cloud-based technologies since they improve their audit efficiencies.

4.6.2. Audit support systems

The continued advancement in artificial intelligence have facilitated the adoption of audit support systems in the audit profession. These systems do not necessarily make decisions but assist auditors to do so (Dillard & Yuthas, 2001). They are perceived to improve audit effectiveness by assisting auditors to make informed professional judgements (Eining et al., 1997; Hunton & Rose, 2010). However, these benefits are not only enjoyed by the mere adoption of these audit support systems, rather they are realized depending on how auditors are reacting to and incorporating these decision aid tools into their decision making process (Fischer, 1996). If auditors view the decision tools as coercive controls, then they are likely to resist using them (Dowling & Leech, 2014) . On the other hand, if they perceive them to be important and improves their decision making process they are likely to adopt them (Dowling & Leech, 2014; Janvrin et al., 2008). Fortunately, auditors generally perceive audit support systems to be helpful in complex decision making situations (Dillard & Yuthas, 2001). As a result, the use of these support systems is still robust with in the auditing profession (Sutton et al., 2016). They increase auditor’s confidence in decision making under conditions of uncertainty (Eining et al., 1997; Ho, 1999).

5. Conclusion

By utilizing a sample of 236 published documents from Internet Related Technology literature, the study analyzed how the research has evolved over the past three decades. The study revealed the most influential classical articles and impactful perspectives in IRT literature which might help researchers to push our knowledge forward. It also reveals the journals which researchers might send their most significant works in future and leading institutions/authors which researchers might collaborate with.

The results revealed that the Journal of Emerging Technologies in Accounting, International Journal of Accounting Information Systems and Journal of Information Systems are the three leading journals in number of publications in IRT literature in the auditing profession. Articles related to big data analytics are the most influential and trending ones. Rugters State University and researchers affiliated to that same institution were identified as the most influential institution and authors in this domain respectively. USA is the most influential country as it provided most of the institutions in the top ten. The results in keywords analyses showed that the last decade witnessed a plethora of keywords such big data, data analytics, cloud auditing and blockchain. Continuous auditing is still a trending key word from authors. The results also show that key words related to audit support systems are fading away especially during the last decade. With the assistance of VOSviewer, the study finds six research streams: (1) the use of big data analytics in the audit profession, (2) impact of Internet Related Technologies on continuous auditing, (3) impacts of Internet Related Technologies on audit quality and efficiency, (4) impact of Internet Related Technologies in fraud detection and risk assessment, (5) blockchain and the audit profession, (6) cloud auditing and audit support systems.

The study contributes to the auditing literature in various ways. Firstly, it provides a detailed review of the intellectual structure of IRT literature over the past three decades. It reveals the six broad research streams which have been explored, the trending and fading keywords. This will assist researchers and the profession at large to have a quick reference guide for a comprehensive overview of the currently available research on IRT. It also uncovers the list of trending and classical articles which normally contains impactful insights. These impactful insights can be a valuable base to predict potential interesting areas for future research. The study further contributes with some interesting future research avenues in Table 7.

Table 7. List of future research questions

Number	Research Stream	Future Research Question	Authors
1	Big Data Analytics (BDA) in the audit profession	1.How to integrate Big Data with traditional evidence?	(Yoon et al., 2015)
		2.What controls can auditors put in place to ensure security of client data in the big data analytics era?	(Huerta & Jensen, 2017)
		3. Under what circumstances should auditors consider Big Data as reliable audit evidence?	(Appelbaum, 2016)
		4. What professional tensions can big data bring between auditors and data scientists? How can we resolve them?	(Salijeni et al., 2019)
2	IRT, risk assessment and fraud	1.How can auditors effectively use social media to detect fraud?	(Dong et al., 2018)
		2. How might auditors deal with noisy data during fraud detection?	(Perols et al., 2017)
3	Effects of IRT on audit quality and efficiency	1.Which phases of the audit are most likely to be affected by IRT?	(Issa et al., 2016)
		2. Under what conditions are auditors most likely to use IRT in judgement decisions?	(Dowling & Leech, 2014)
5	Impacts of IRT on Continuous Auditing.	1. How will blockchain technology change the continuous auditing process?	(Issa et al., 2016)
		2. What knowledge should auditors acquire to be able to use blockchain technology in continuous auditing systems?	(Dai & Vasarhelyi, 2017)
		3. How can blockchain mechanism be integrated into the continuous auditing system?	(Dai & Vasarhelyi, 2017)
		4. What type of new evidence will be brought into continuous auditing systems by big data?	(Zhang et al., 2015)
		5. How can this type of evidence be integrated into continuous auditing systems?	(Zhang et al., 2015)
6	Cloud Auditing and audit support systems	1.What are the empirical benefits of cloud auditing?	(Chou, 2015)
		2.What is the status quo of cloud auditing adoption in auditing firms?	(Chou, 2015)
7	General	1.How might auditors, who lack IRT skills and knowledge be best trained?	(Issa et al., 2016)
		2. What are the effects of IRT on auditing firm’s recruitment strategies?	(Issa et al., 2016)

The study also provides some practical implications for the IRT literature in auditing. The results revealed that literature is concentrated in journals in ABS level 1 and 2. This calls for a shift from the conceptual type of studies currently dominating the literature to empirical research which is theoretically driven focusing on the actual implementation of IRT in the audit profession. By so doing, researchers can test and refine IRT propositions, concepts, theories and ultimately enhancing the chances of publishing in highly ranked journals. Secondly, the literature on IRT is dominated by USA researchers. This calls for researchers from other geographical parts of the world to also venture into this research domain and provide a diversified perspective of IRT implementation in the audit profession.

The results suggest that majority of influential researchers are from or were once affiliated to the Continuous Audit and Reporting Laboratory (CarLab) at Rutgers State University. Researchers from other geographical locations can form similar specialized research institutions. These institutions will foster specialization on experienced faculty heading them and nature a plethora of emerging scholars with an avid interest in this research domain through various PhD programs. Collaborations within and amongst these research institutions must be encouraged as this stimulates cocreation of knowledge. As a starting point, these institutions may target conference proceedings, where they will receive more insights about their works and try to introduce themselves to the IRT community. However, one main drawback of conference proceedings is they normally do not receive high citations compared to published work in highly ranked peer reviewed journals. Therefore, conference proceedings might be used as a steppingstone to improve their research and publish in peer reviewed journals. This will improve the quality and volume of the research institution’s production, ultimately offering the much-needed diversified geographical perspective of IRT adoption in the auditing profession.

Despite these interesting contributions to the IRT literature, one possible limitation of this study is its reliance on bibliography from the WOS. Even though WOS is one of the most widely used data base in bibliometric reviews due to its journal selectiveness (quality and impact) and compatibility with several bibliometric software, it does not incorporate all IRT related publications. Thus, articles in journals which are not registered on WOS but registered on Scopus such as The International Journal of Digital Accounting Research, were missed. Indeed, Scopus has a slightly wider journal coverage than WOS. However, the omission of bibliometrics information from Scopus dataset was primarily due to several reasons. To begin with, one of the software used in the study (HistCite) is not compatible with bibliometric data from Scopus. By opting for Scopus, the study was going to be deprived of the valuable citation analyses obtainable from HistCite. More so, author indexing is poorer in Scopus as compared to WOS. That is, the same author can be indexed differently. This might create author duplication problems when running citation and author collaboration analyses. Lastly, WOS is selective and journals indexed in it are of higher quality than the journals indexed in Scopus. Another limitation of the study pertains to the timeframe constraints since the bibliography used is from 1990 to 2019. Finally, the study was biased towards publications written only in English.