Issue 4open Volume 5, 2022 Statistical Inference in Markov Processes and Copula Models 20 9 Mathematics - Applied Mathematics https://doi.org/10.1051/fopen/2022022 21 December 2022 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

## Introduction

In this article, we address the notion of conditional independence and its impact on unconditional dependence. Here, unconditional dependence is being represented by a copula function and the impact of conditional independence on unconditional dependence is studied using the assumption of conditional independence in the copula function. Independence allows us to carry out the usual inferential process, for example, those obtained by calculating the likelihood function, and because of that is a relevant notion in statistical inference. Under the independence assumption is possible to carry out the classic techniques of point estimation, hypothesis testing, and all those procedures based on the likelihood function. As can then be verified, the concept of independence occupies a relevant space in statistics and for this reason, it is widely found in the literature. Several efforts have been done to promote its identification, see for instance García and González-López [1, 2]. In such papers are proposed statistics for independence detection, see also Genest and Rémillard  and Hollander et al. . Such statistics work under certain conditions imposed on the nature of the underlying variables. Other authors are concerned with measuring dependence. For example, García et al.  proposes an index with such purpose. Despite its enormous use, independence is a very restrictive case within the diversity of types of dependencies that could occur. For example, if we consider the random vector (X, Y) with bivariate Normal distribution, and density function then, the independence only occurs when ρ = 0, which shows that independence is a restrictive assumption. The theory of copulas has been extensively used to investigate such diversity. Nelsen  and Joe  show the enormous diversity of possible dependencies, addressed from the notion of copulas.

This article explores conditional independence, seeking to preserve some inferential procedures, and we also seek to characterize the impact of conditional independence on dependence. Conditional independence requires the identification of the entity Θ under which we can declare conditional independence (conditional to Θ), that requirement can be a constraint in practice since there is no clear identification procedure of Θ. The first question we seek to answer is how conditional independence impacts unconditional dependence. For the characterization of unconditional dependence, we will rely on the notion of copula. Secondly, we approach the impact of the notion of infinite exchangeability on conditional independence, based on the fact that such a notion guarantees conditional independence [8, 9]. With such a notion in hand, we return to the investigation of the structure of the copula representing unconditional dependence. That is, in this paper, we investigate the impact of conditional independence on unconditional dependence and extend this question to a stronger notion as is the case of infinite exchangeability.

Independence is a very restrictive case within the diversity of types of dependencies, on the other hand, conditional independence (to Θ) is a much more flexible concept, and for this reason, it is the condition that we explore in this article. In our approach, we take the entity Θ as being a random variable since in this way we generalize the notion. In addition, we give space so that the reader can access the interpretation that de Finetti offers of such an entity.

Here we describe the content of this paper. In Section 2 we address the notion of conditional independence and we introduce a theorem that shows a representation of the dependence (copula). We also show corollaries addressing situations of interest that result from that theorem. In Section 3 we introduce the notion of infinite exchangeability, which allows us to identify the quantity Θ, guarantying conditional independence. We then show a result offering a representation of the dependence (copula density function), given the property of infinite exchangeability. The final considerations are given in Section 4.

## Conditional independence

We begin this section by introducing the notion of conditional independence and then exemplifying it. We introduce in the sequence the notion of copula function. From these notions follows our main result, Theorem 2.1 which shows the impact of conditional independence on the dependence, that is, in this case, on the copula structure. We emphasize the consequences that result from Theorem 2.1 in two corollaries and we show also some examples.

## Definition 2.1

Given two continuous random variables X and Y, X and Y are conditionally independent given Θ, where Θ is a random variable, if with and denoting the conditional density functions, given θ, of (X,Y), X and Y respectively.

The next example shows a case of conditional independence given Θ.

## Example 2.1

Set the joint density function of X, Y, Θ, as where IA (x) = 1, if x ∈ A, and IA (x) = 0, if x ∉ A. The marginal distributions fX,Y, fX, fY and fΘ are respectively,  , and Moreover, then X and Y are conditionally independent given Θ = θ, with different conditional distributions, being identically distributed given θ, when k = 1.

From the previous example we see that conditional independence does not impose the same conditional distribution. Note that Definition 2.1 allows θ to be considered as a constant value, and in such case θ is a parameter.

The result that we show in Theorem 2.1 seeks to understand the meaning that Θ plays in the construction of the dependence between X and Y (not conditioned to Θ = θ). For dependence between X and Y we understand the copula between them. In rough terms, the copula is a function that, combined with the marginal cumulative distributions, returns the joint cumulative distribution.

Next we formally introduce the notion of copula explaining how this notion serves to represent the dependence between X and Y.

## Definition 2.2

A copula (in dimension 2) is a function C: [0, 1]2 → [0, 1] with the following properties,

1. u,v ∈ [0, 1], C(u,0) = C(0,v) = 0, and C(u,1) = u,C(1,v) = v;

2. for every ui, vi ∈ [0, 1], i = 1, 2, such that and  It is possible to show that every copula given by Definition 2.2 is a cumulative distribution function on [0, 1]2, in the strict sense of the term. We now show how Definition 2.2 is formally related to the dependence between the variables of a random vector.

According to Sklar’s theorem , if HX,Y is the joint distribution function of (X, Y) with margins FX and FY, there exists a copula CX,Y (called 2-copula of (X, Y)) such that  If FX and FY are continuous, then CX,Y is unique. Conversely, if C is a copula (following Definition 2.2), F and G are distribution functions, then the function H(x,y) = C(F(x),G(y)) is a joint distribution function with margins F and G.

From what is given in Sklar’s theorem, the dependence between X and Y is represented by the 2-copula of (X,Y), CX,Y, since the margins FX and FY depend on X and Y, respectively. Then, our purpose is to see the effect of Definition 2.1 in CX,Y.

The next theorem provides an analytical representation of the copula, knowing that the variables are conditionally independent given the entity Θ.

## Theorem 2.1

Given X and Y under the assumptions of Definition 2.1, if CX,Y is the 2-copula of (X,Y), Θ the random variable of Definition 2.1 with density function πΘ, (1)

with CX and CY denoting the 2-copulas of (X,Θ) and (Y,Θ) respectively, and FΘ denoting the cumulative distribution of Θ.

Proof. Since X, Y are continuous random variables we can write the cumulative distribution HX,Y between X and Y as follows, applying Definition 2.1, (2)

Considering the cumulative distribution between X and Θ, by Sklar’s theorem, HX(s,θ) = CX (FX(s), FΘ(θ)) then, the joint density function between X and Θ is given by (3)

with cX denoting the density function related to the copula CX. Then, from equation (3), we obtain, (4)

□We proceed to integrate the equation (4). Define u = FX(s), then, du = fX (s)ds and (5)

We have similar expressions for fY(t,θ) and fY|θ(t). Then, applying equation (5) in equation (2), we obtain (6)

Consider u and v in [0, 1], there exist values x and y, such that u = FX(x) and v = FY(y), then, by Sklar’s theorem and equation (6), □

We use in the following example a family of copulas that corresponds to weak dependence. It is the Farlie–Gumbel–Morgenstern family, and it encompasses the case of independence. See Nelsen .

## Example 2.2

Consider (X,Y) with a 2-copula (7)

with parameters α ∈ [−1, 1] and β ∈ [−1, 1]. Then, (X,Y) has a Farlie–Gumbel–Morgenstern 2-copula with parameter Consider Θ as a continuous random variable, with Uniform distribution in (0,1).

If we set the 2-copulas of (X, Θ) and (Y, Θ) as being Farlie–Gumbel–Morgenstern 2-copulas with parameters α ∈ [ 1, 1] and β ∈ [−1, 1], respectively. Since θ ~ U(0,1), we have And, we obtain, Then, As a consequence, One question that arises is whether every expression given by the right side of equation (1) is a copula, where CX and CY are copulas. The Theorem 1, in González-López and Litvinoff Justus , states that the expression is always a copula. That is, postulating two copulas of (X, Θ) and (Y, Θ), we get a copula that could be used to deal with the dependence between X and Y. An example of what is stated in this paragraph appears in the previous example, since taking (X, Θ) and (Y, Θ) with Farlie–Gumbel–Morgenstern copulas, with parameters α and β, respectively, the resulting copula between X and Y (using the right side of equation (1)) is a Farlie–Gumbel–Morgenstern copula, with parameter The following corollary shows the version of Theorem 2.1 in terms of density functions of copulas.

## Corollary 2.1

Under the assumptions of Theorem 2.1, if cX,Y, cX and cY are the density functions related to the 2-copulas CX,Y, CX and CY, respectively, (8)

Proof: Consider that  and The results follow differentiating both sides of equation (1) and permuting the integral with □

## Example 2.3

Considering the Example 2.2. Since  Then, (9)

and, according to the equation (7), it is the copula density function of a FGM copula with parameter The following corollary shows the version of Theorem 2.1 under the assumptions of identical conditional margins. The impact of the assumption appears in the integrating term of equation (1). This assumption about margins will become relevant later, in this paper.

## Corollary 2.2

Under the assumptions of Theorem 2.1, if the variables X and Y are conditionally (to Θ) independent and identically distributed, ## Example 2.4

Considering the Example 2.2, if α = β, the copula of (X,Y) is a Farlie–Gumbel–Morgenstern copula with Then, the restriction (on the margins) imposes on the relationship between X and Y a positive Kendall’s τ coefficient .

Theorem 2.1 shows that the conditional independence of X and Y given Θ defines the structure of the copula between X and Y. See equation (1) for the form of the copula and see Corollary 2.1 for the form of the density function of copula between X and Y. We exemplify both results in Examples 2.2 and 2.3, respectively. Corollary 2.2 shows the case covered by Corollary 2.1, under the assumption of identical margins. The latter is exemplified in Example 2.4. The Corollary 2.2 is especially interesting for the situation that we will address in the next section.The results of this section describe the analytical form of the copula between X and Y, no longer depending on Θ. This means we have offered a representation for the predictive copula between X and Y.

The key to determining conditional independence between X and Y is to know that there exists a variable Θ establishing this independence. This leads us to the following reflections, showing a context where such an entity can be identified. That is the goal of de Finetti’s theorems of representation.

## Exchangeability

We start the section with the definition of infinite exchangeability. This notion is weaker than independence and makes it possible to identify Θ. Then, we exemplify certain characteristics that result from this imposition. And, we present in Corollary 3.1, the impact of this notion on the dependence structure, that is, on the copula function.

## Definition 3.1

A sequence X1,X2,X3,... of random variables is infinitely exchangeable, if for each collection of n variables the joint distribution is identical for all the permuted set of variables with σ: {1, 2,…,n} → {1, 2,…,n} any permutation function.

The success in identifying the Θ entity stems from de Finetti’s representation theorem, see de Finetti . de Finetti  proves that, a binary sequence X1, X2,... follows Definition 3.1, if and only if, there exists a distribution function FΘ on [0, 1] such that for all n the joint probability mass function where FΘ is the cumulative distribution function of the entity and xi ∈ {0, 1}. As a consequence, the conditional mass probability is decomposed in a product of Bernoulli’s with being θ a value of the variable Θ. Such a result is extended for all types of random variables in Hewitt and Savage . For a sequence X1,X2,... of continuous random variables following Definition 3.1, there exists a variable Θ, such that the conditional density function can be decomposed as The previous relation happens for each value θ of the variable Θ, then, the structure imposed by Definition 3.1 in the sequence X1,X2,⋯ guarantee the existence of the entity Θ.

We see in the remark to follow (Remark 3.1) that this notion leads us to conditional independence, and implies identical margins. But as we show in Example 3.1, exchangeability coexists with dependence.

## Remark 3.1

Infinitely exchangeable variables (following Definition 3.1) are identically distributed conditionally to θ (value of the variable Θ identified by de Finetti’s theorem). If the infinitely exchangeable sequence is composed by absolutely continuous random variables and Θ has a density function πΘ (·), for each as a consequence, since, The concept of infinite exchangeability could be considered unrealistic, but it can be seen applied in usual inferential strategies. If a sample is available, say X1,,Xn, when constructing the likelihood function its elements are considered independent given a certain parameter. And how does this assumption is related to Definition 3.1? If the distribution of X1,…,Xn is invariant under permutations of the elements of X1,…,Xn, then we say that X1…,Xn is exchangeable. If X1,…,Xn is exchangeable , then, we say that X1, X2,… is infinitely exchangeable. If X1,…,Xn can be embedded in an infinitely exchangeable sequence X1, X2,…, then we say that X1,…,Xn is infinitely exchangeably extendable. Bayesian and frequentist approaches, in general, treat observable values as infinitely exchangeably extendable and, as a consequence of de Finetti’s representations, independent and identically distributed conditional on some unknown entity Θ.

Although identically distributed independent random variables must be exchangeable, identically distributed exchangeable random variables need not be independent. Consider for instance the bivariate case of Gumbel distribution as shown by the next example.

## Example 3.1

Let (X,Y) be a random vector, H its joint distribution function, given by and H(x,y) = 0, otherwise, with α a parameter in [0, 1]. We see that X and Y are exchangeable. Note also that X and Y have identical marginal distribution (exponential distribution). But, since the 2-copula of (X,Y) is X and Y are not independent.

The previous example shows us a case where exchangeability occurs, the variables share the same marginal distribution, but X and Y are not independent. Thus we see that exchangeability is a distinctly different notion from the notion of independence. It also follows from Example 3.1 that identically distributed variables can be dependent since every copula is a joint distribution with uniform marginal distributions, in particular, the copula of Example 3.1.

The next example shows a case of dependent variables, with identical marginal distributions but not exchangeable.

## Example 3.2

Let (X,Y) be a random vector, H its joint distribution function, given by H(x,y= xy − x3y(1 − x)(1 − y), x,y ∈ [0, 1]. Then, H is a copula, since fits the conditions of Definition 2.2. X and Y are identically distributed but, not exchangeable since H is not symmetrical.

Bearing in mind the previous example, it is now necessary to mention Theorem 2.7.4  which shows how the margins should be and also the copula to guarantee the exchangeability between a pair of variables. Let X and Y be continuous random variables with joint distribution function H, margins FX and FY, respectively, and copula C. Then X and Y are exchangeable if and only if FX = FY and C is symmetric (C(u,v) = C(v,u), ∀(u,v) ∈ [0, 1]2).

In the treatment and modeling via copulas, it is very common to operate with the original variables transformed by their margins, so below we summarize the impact of Definition 3.1 on such variables.

## Remark 3.2

A property that is straight from Definition 3.1, is the preservation of the infinite exchangeability from the original sequence to the standardized ones. Namely, if the sequence X1, X2, X3,… of random variables is absolutely continuous, the sequence U1, U2, U3,… is also infinitely exchangeable, where Since, for any collection i1,…in, with arbitrary n, for any permutation σ and arbitrary values u1,…,un ∈ [0, 1].  (10) (11)

where equation (10) results from Definition 3.1 and equation (11) follows from Remark 3.1 ( ).

We note that if the variables U1, U2, U3,… of Remark 3.2 are infinitely exchangeable, this does not imply that the variables X1, X2, X3,… are infinitely exchangeable. It is enough that one of the marginal distributions is different from the rest which would violate Remark 3.1.

Under the framework of Definition 3.1 and assuming the continuity of the variable identified by the de Finetti representation, say Θ, we will have the copula between pairs of variables of the infinitely exchangeable sequence given by the copula between a member of the sequence (the first one is enough) and the variable Θ, let’s see.

## Corollary 3.1

Let X1, X2,… be a sequence of continuous random variables following Definition 3.1. If Θ is the variable identified in the de Finetti representation, producing the conditional independence, and Θ has density function πΘ(·), then, for each pair of variables Xi and Xj, i ≠ j, (12)

where and are the density functions related to the 2-copulas and respectively.

Proof. Since infinitely exchangeable variables are conditionally independent (to the random variable of de Finetti’s theorem) for each pair Xi and Xj (from Corollary 2.1).

Infinitely exchangeable sequences of absolutely continuous random variables share their unconditional marginal distributions from Remark 3.1. Also, for each θ value of Θ, since, (13)

where equation (13) follows from Sklar’s theorem. And, using that we can see that □

However, a question that arises is whether there is any restriction on the type of copula that could result from Corollary 3.1, a hint is given in O’ Neill , formulated in the next remark.

## Remark 3.3

Let X1,X2,… be a sequence of continuous random variables following Definition 3.1. Then, from Theorem 2 of O’ Neill , for each pair of variables Xi and Xj, i ≠ j, i, j ≥ 1, with denoting the Pearson’s rho correlation coefficient. Then, from Remark 3.2, since the variables U1,U2,… follow Definition 3.1, for each pair of variables Xi and Xj, i ≠ j, i, j ≥ 1, with denoting the Spearman’s rho correlation coefficient.

The above remark indicates that the possible copulas identified in Corollary 3.1 cannot correspond to negative correlations, under Definition 3.1.

The following example shows a case under Definition 3.1, and the resulting predictive copula.

## Example 3.3

Let X1, X2,… be an infinite sequence, under Definition 3.1, of exponential random variables with parameter θ, conditionally independent, given θ, a value of the continuous random variable Θ with exponential distribution and hyperparameter λ. Then, each conditional density function is and the prior density function of Θ is  is proportional to for Since, for and Using the previous proportionality we obtain On the other hand, for then, the right side of equation (12) is proportional to Verifying the Corollary 3.1.

Then, since we have the copula density function  and the predictive 2-copula is We start the section with some notes and examples so that Corollary 3.1 is a natural consequence of the impact of conditional independence in the framework of infinite exchangeability. Remark 3.1 shows that infinite exchangeability implies conditionally independence and identical margins. While independence implies permutability, the opposite is not valid, see Example 3.1. Also, if the margins are all equal this does not imply permutability, see Example 3.2. Under Definition 3.1, we show in Corollary 3.1 that infinite exchangeability allows to obtain a representation of the copula between pair of elements of the infinite sequence in terms of the copula between each element of the infinite sequence and the entity Θ, quantity identified by de Finetti’s theorem. Also, note that the right side of equation (12) reflects that is only necessary to know the copula between one element of the infinite sequence and Θ to obtain the predictive copula between pairs of elements of the infinite sequence. Example 3.3 exposes how works in practice the corollary.

## Conclusion

In the framework of absolutely continuous random variables X and Y, we use the notion of conditional independence between X and Y (conditional on Θ), to obtain a representation of the predictive 2-copula between X and Y (Theorem 2.1, see also Corollary 2.1). Such a result tells us that the representation involves the 2-copula between X and Θ, the 2-copula between Y and Θ, and the distribution of Θ. Being the property of identically distributed useful for inferential procedures, in Corollary 2.2 we reveal the impact of such an assumption on the representation of the predictive 2-copula between X and Y. There are indications (see Example 2.4) that the restriction imposed by the condition of identical margins could lead to restrictions on the values of coefficients of dependence, such as Kendall’s τ coefficient, Spearman’s ρ coefficient.

Since the notion of conditional independence requires the identification of the entity Θ making X and Y conditionally independent given Θ, we introduce the concept of infinite and exchangeable sequences, X1, X2, X3.... This condition appears as one of the inferential strategies applicable in the process of manipulating finite samples. It is reasonable to suppose that a finite sequence is part of an infinite sequence. Under Definition 3.1, by representation theorems [8, 9] we can guarantee that Θ exists and that it is a random variable. To give the reader a more precise notion of the meaning of infinite exchangeability, we show that it imposes the same margins on X and Y (conditional or not, see Remark 3.1), but that it does not impose unconditional independence (see Example 3.1) and it does impose conditional independence on Θ. Then, under infinite exchangeability, we revisit the copula density function structure and prove Corollary 3.1, which shows the structure of the predictive 2-copula density function between Xi and Xj, i ≠ j elements of the infinite sequence. We complete our findings with examples and notes for the reader. Corollaries 2.2 and 3.1, although they start from different assumptions, show that the predictive density function of the 2-copula between pairs of variables X and Y can be analytically expressed as a function of the density functions of the copula between only one variable (say X) and Θ. That is to say that the dependence, represented by the predictive 2-copula between X and Y, is only a consequence of the dependence between X and Θ. This fact shows the relevance of the identification of Θ, for the description of the predictive 2-copula.

## Acknowledgments

Vinícius Litvinoff Justus gratefully acknowledges the support provided by CNPq with a fellowship of scientific initiation from University of Campinas. The authors wish to thank the referees and editors for their many helpful comments and suggestions on an earlier draft of this paper.

## References

1. García JE, González-López VA (2020), Random permutations, non-decreasing subsequences and statistical independence. Symmetry 12, 9, 1415. https://doi.org/10.3390/sym12091415. [CrossRef] [Google Scholar]
2. García JE, González-López VA (2014), Independence tests for continuous random variables based on the longest increasing subsequence. J Multiv Anal 127, 126–146. https://doi.org/10.1016/j.jmva.2014.02.010. [CrossRef] [Google Scholar]
3. Genest C, Rémillard B (2004), Test of independence and randomness based on the empirical copula process. Test 13, 335–369. https://doi.org/10.1007/BF02595777. [CrossRef] [Google Scholar]
4. Hollander M, Wolfe D, Chicken E (2013), Nonparametric statistical methods, John Wiley & Sons, New York. [Google Scholar]
5. García JE, González-López VA, Nelsen RB (2013), A new index to measure positive dependence in trivariate distributions. J Multivar Anal 115, 481–495. https://doi.org/10.1016/j.jmva.2012.11.007. [CrossRef] [Google Scholar]
6. Nelsen RB (2006), An introduction to copulas, Springer-Verlag, New York. https://doi.org/10.1007/0-387-28678-0. [Google Scholar]
7. Joe H (2014), Dependence modeling with copulas, Chapman and Hall/CRC, New York. https://doi.org/10.1201/b17116. [CrossRef] [Google Scholar]
8. De Finetti B (1929), Funzione caratteristica di un fenomeno aleatorio, in: Atti del Congresso Internazionale dei Matematici: Bologna del 3 al 10 de settembre di 1928 (Comunicazioni, sezione IV (A)–V–VII), vol. 6, pp. 179–190. [Google Scholar]
9. Hewitt E, Savage LJ (1955), Symmetric measures on Cartesian products. Trans Am Math Soc 80, 2, 470–501. https://doi.org/10.2307/1992999. [CrossRef] [Google Scholar]
10. González-López VA, Litvinoff Justus V, A method for the elicitation of copulas (preprint) [Google Scholar]
11. O’ Neill B (2009), Exchangeability, Correlation, and Bayes’ Effect. Int Stat Rev 77, 2, 241–250. https://doi.org/10.1111/j.1751-5823.2008.00059.x. [CrossRef] [Google Scholar]

Cite this article as: González-López VA & Litvinoff Justus V 2022. Conditional independence and predictive copula. 4open, 5, 20.

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.